System Events

This document describes the Kafka messages that the system emits internally, and their consumers. All messages are encoded as JSON objects.

TOPIC: node.email

This message is emitted when a user needs to be notified via email. There are multiple services that send it, and independent email sender is listening for these messages just to render the text, compose the email, and send it.

  • datetime – the original timestamp when the email was initiated
  • template_id – the name of the email template
  • template_vars – data for the template.
  • lang_code – the language of the e-mail (defaults to English)
  • recipient_name – the name of the recipient
  • recipient_email – the email address where to send
  • sender_name – the name of the sender
  • sender_email – the email address of the sender (or reply-to)
  • retry_count – the number of the attempt (initially: 0).

TOPIC: node.user

The message is emitted for user info changes. Although, generally this topic records user-events for historical purposes, it is also used for disconnecting users from the application when their session is revoked due to a role-change.

  • target – the username that was affected.
  • parent – the parent organisation (ID).
  • scope – the type of change with potential values:
    • profile – user’s name and description change, or when user is added.
    • roles – user’s role change.
    • session – user’s session started or ended.
  • before – the value before (session ID when session ended); can be omitted.
  • after – the value after (session ID when session started); can be omitted.

TOPIC: node.session

The message is emitted for user-session events that are initiated by the user itself (including log-out due to a new login). Currently, this covers only log-out scenarios for closing open WebSocket connections.

  • session_id – the session identifier that this event applies to.
  • username – the username of the session.
  • action – the type of action (currently only the end value is used).

TOPIC: dataset.<state>

This is a short-hand topic-description for each state of the dataset. The full list of topics covered here is:

  1. dataset.unregistered
  2. dataset.registered
  3. dataset.public
  4. dataset.hidden
  5. dataset.archived

The message is emitted when a new dataset is detected and added to the database (dataset.unregistered) or when a dataset is moved to one of these states.

  • dataset_id – internal numeric ID of the dataset.
  • dataset_pid – public persistent identifier (once dataset is public).
  • updated_by – the username of the user who updated this dataset. When the system itself triggered it, it’s value is gdi-node-pipelines.

TOPIC: dataset.deleted

This message is emitted when a data-provider user requests the deletion of a dataset, which is in the REGISTERED or UNREGISTERED state. The message triggers a pipeline, which removes dataset files from S3 storage, and its database records. It does not remove related Kafka messages.

  • provider_id – reference to the owner (ID) of the dataset.
  • dataset_id – internal numeric ID of the dataset.
  • user_id – the username of the user who requested the deletion.