Tutorial

This tutorial takes you through the process of creating, sharing, updating, and retrieving records on the BIG-MAP Archive.
Before you start, check that you have a user account on the demo instance of the data repository, which enables you to try available features and commands without affecting the main instance. To get an account on the demo instance contact us at big-map-archive@materialscloud.org.

Data sharing via the User Interface

This section shows you how to create, share, update, and retrieve records using the user interface of the demo instance.

Log in to the demo instance

Navigate to the homepage.
Click "Log in" and enter your account credentials. If you receive an error message such as "Invalid email address", contact us.
In case of unknown or forgotten password:
- Click "Forgot password?".
- Enter your email address and click "Reset password".
- Follow the instructions. Choose a new password that must be at least 6 characters long.
- Upon successful login, you are automatically redirected to the shared records' first page.

Create a record

Click "New record" in the header.
Community (compulsory to share with a community): select the community with which you want to share the record.
Note that if you select BATTERY2030+ the record will be shared with the members of all BATTERY2030+ projects.
Files (compulsory): select the files to upload and link to the record. Drag and drop them in the drop zone (light gray region). Note that you are limited to 100 files and a total file size of 100 GB. Moreover, for testing, we recommend choosing small files (file size ≤ 1 MB) containing dummy data. Note that the data will be accessible to any demo user, so do not upload any confidential data and consider that data on the demo instance can be deleted at any time (and we will periodically delete data).
Fill in the record metadata:
- A resource type (compulsory), it describes the overall nature of your data files’ contents.
- A title (compulsory).
- Authors (compulsory). Note that adding another users as authors does not make them record's co-owners.
- A description (compulsory).
- A license (optional). The recommended license is the BIG-MAP license which allows re-distribution and re-use of work within the BIG-MAP community.
- Keywords (optional). Any keyword is allowed. However, each keyword should be entered separately: enter the keyword and click on Add <keyword>. To delete the keyword click on the small cross on the right side of each of keyword. You also benefit from an autocomplete feature that suggests keywords based on input. Type, e.g., "wp". Recommendation: if you split a large dataset over multiple entries, you may want to tag the first version of each entry with the same unique identifier. This might help humans and machines to retrieve the whole dataset by using the built-in search engine and searching for that identifier.
- References to external documents (optional).
Click "Save draft" to create a draft, i.e., the first version of the entry. It remains visible only to you (owner).
Click on "Preview" to see how the record will display once it is shared with a community.

Share the record with a community

Go to "My records" and click the record title.
Check that you already selected a community; if not, click on "Select community".
Click "Share with community" to make your record appear in the Shared records list. By sharing the record, you give all members of the chosen community permission to read the record metadata and download its linked files.
Note that if you selected BATTERY2030+, the record will be shared with the members of all BATTERY2030+ partner projects.

Update the record (metadata only)

After the record has been shared, you can only update the metadata of the record. The community and the record files cannot be changed.

Search for your record in the search bar or in "My records", click the record title and then click on "Edit".
Make the changes: e.g. add a keyword, an author, etc.
Click on "Save draft" to save the changes. The updated record will only be visible to you. Click on "Share with community" to share the updated record with the community.
Navigate to "Shared records" and click the record title. Your modifications should be visible to the members of the community you selected.

Create and share a second version of the record

Note that the community of a new version is the same community that you selected for the first version and cannot be changed.

Search for your record in the search bar or in "My records", click the record title and then click on "New version".
Click "Import files" to import all file links from the first version into the second one.
Remove files, upload new files, change the title, and in general apply all needed modifications.
Click on "Save draft" to save the new version as a draft. It will only be visible to you. Finally, click on "Share with community" to share the new version with the community.
Navigate to "Shared records". The second version should appear in the shared records list, as only the latest shared version of an entry is displayed by default.
Switch the toggle on the left side of the web page to show both versions.
Click on any of the two versions. Find the card entitled "Versions" on the right side of the web page. You should be able to easily navigate from one version to the other by clicking the appropriate link.

Share a record with a link

Create and share a link to let others read or edit your record.

Search for your record in the search bar or, if the record is a draft, go to "My records" and click the record title.
Click on "Share links" and follow the instructions to create a link with the permissions to view, preview or edit the record.

Search through shared records metadata

Click on "Shared records".
Type, for instance, "+LNO +WP9" into the search bar to search for all shared records whose metadata contains LNO and WP9. Depending on the contents of the database, the built-in search engine returns zero or more hits. For more advanced queries, such as searching through specific fields in the metadata (e.g., the title), read the provided search guide.

Data sharing via the APIs

In this section, users interact with the data repository through a command-line interface (CLI) application, which is installed on their machine, via a REST API that is provided by the BIG-MAP Archive. Commands to create, update, and retrieve records can be executed at terminal prompts and even invoked from custom applications. Note that more complex commands can be built upon those presented below to meet special user needs.

Get an API token

Login to the Archive and click over your email on the right side of the web page. Select "Applications" in the drop-down menu.
Click "New token" and follow the instructions to create an API token with unlimited lifetime. Keep it private and in a safe storage. You will use it to access API endpoints of the data repository.

Get started with the CLI app

Follow the Quick start guide to:

install the CLI app on your machine
prepare a YAML configuration file named `bma_config.yaml` that specifies the domain name and your API token for the data repository
prepare a YAML file named `metadata.yaml` that contains your future record metadata (its title, its list of authors, etc).

Activate the virtual environment where the package `big-map-archive-api-client` is installed.
Test that the following command is available:

Create and share (a first version of) an entry

Move `metadata.yaml` to the `data/input` directory.
Put the data files to be uploaded and linked to the future record in the `data/input/upload` directory.
To publish a record to a community you need to insert the community slug, e.g. bigmap, to publish on the BIG-MAP community. On the demo instance get the list of your communities and their slug at https://archive-demo.big-map.eu/api/communities.
Execute the following command:
Once this is done, access the record web page by simply clicking the generated hyperlink in the terminal.
Store the record id.

Update the first version (metadata only)

Update the contents of `metadata.yaml` (e.g., the title).
Execute the following command, after replacing `<record_id>` with the id of the first version:
Check that your modifications appear as expected in the record web page.

Create and share a second version

Update the contents of the `metadata.yaml` file and the `upload` directory:
- Modify the description.
- Change the contents of a file.
- Remove a file.
- Add a new file.
Execute this command, after replacing `<record_id>` with the id of the first version:
Note that, if a data file remains unchanged from one version to the next, it is linked to both versions but uploaded only once. This saves storage space and avoids unnecessary use of bandwidth.
Navigate to "Shared records" and click the record title. Observe the links imported from the first version, in addition to the links for the files in `data/input/upload`. The `--link-all-files-from-previous` flag is responsible for these extra links.

Retrieve shared records' metadata

To retrieve the metadata of a specific shared record, execute this command, after replacing `<record_id>` with the record's id:
To get the metadata of the latest shared version of each entry, execute this command:
For all shared versions of each entry: