Snowplow is an enterprise-grade marketing and Product Intelligence platform that tracks how users engage with our website and application.

Snowplow consists of several loosely-coupled sub-systems:

  • Trackers fire Snowplow events. Snowplow has twelve trackers that cover web, mobile, desktop, server, and IoT.
  • Collectors receive Snowplow events from trackers. We use different event collectors that synchronize events to Amazon S3, Apache Kafka, or Amazon Kinesis.
  • Enrich cleans raw Snowplow events, enriches them, and puts them into storage. There is a Hadoop-based enrichment process, and a Kinesis-based or Kafka-based process.
  • Storage stores Snowplow events. We store the Snowplow events in a flat file structure on S3, and in the Redshift and PostgreSQL databases.
  • Data modeling joins event-level data with other data sets, aggregates them into smaller data sets, and applies business logic. This produces a clean set of tables for data analysis. We use data models for Redshift and Looker.
  • Analytics are performed on Snowplow events or on aggregate tables.


Enable Snowplow tracking

Tracking can be enabled at:

  • The instance level, which enables tracking on both the frontend and backend layers.
  • The user level. User tracking can be disabled on a per user basis. GitLab respects the Do Not Track standard, so any user who has enabled the Do Not Track option in their browser is not tracked at a user level.

Snowplow tracking is enabled on, and we use it for most of our tracking strategy.

To enable Snowplow tracking on a self-managed instance:

  1. On the top bar, select Menu > Admin, then select Settings > General. Alternatively, go to admin/application_settings/general in your browser.

  2. Expand Snowplow.

  3. Select Enable Snowplow tracking and enter your Snowplow configuration information. For example:

    Name Value
    Collector hostname
    App ID gitlab
    Cookie domain
  4. Select Save changes.

Snowplow request flow

The following example shows a basic request/response flow between the following components:

    participant Snowplow JS (Frontend)
    participant Snowplow Ruby (Backend)
    participant Snowplow Collector
    participant S3 Bucket
    participant Snowflake DW
    participant Sisense Dashboards
    Snowplow JS (Frontend) ->> Snowplow Collector: FE Tracking event
    Snowplow Ruby (Backend) ->> Snowplow Collector: BE Tracking event
    loop Process using Kinesis Stream Snowplow Collector ->> Snowplow Collector: Log raw events Snowplow Collector ->> Snowplow Collector: Enrich events Snowplow Collector ->> Snowplow Collector: Write to disk
    end Snowplow Collector ->> S3 Bucket: Kinesis Firehose
    Note over Snowplow Collector, S3 Bucket: Pseudonymization
    S3 Bucket->>Snowflake DW: Import data
    Snowflake DW->>Snowflake DW: Transform data using dbt
    Snowflake DW->>Sisense Dashboards: Data available for querying

Structured event taxonomy

Click events must be consistent. If each feature captures events differently, it can be difficult to perform analysis.

Each click event provides attributes that describe the event.

Attribute Type Required Description
category text true The page or backend section of the application. Unless infeasible, use the Rails page attribute by default in the frontend, and namespace + class name on the backend.
action text true The action the user takes, or aspect that's being instrumented. The first word must describe the action or aspect. For example, clicks must be click, activations must be activate, creations must be create. Use underscores to describe what was acted on. For example, activating a form field is activate_form_input, an interface action like clicking on a dropdown is click_dropdown, a behavior like creating a project record from the backend is create_project.
label text false The specific element or object to act on. This can be one of the following: the label of the element, for example, a tab labeled 'Create from template' for create_from_template; a unique identifier if no text is available, for example, groups_dropdown_close for closing the Groups dropdown in the top bar; or the name or title attribute of a record being created.
property text false Any additional property of the element, or object being acted on.
value decimal false Describes a numeric value (decimal) directly related to the event. This could be the value of an input. For example, 10 when clicking internal visibility.


Category* Label Action Property** Value
[root:index] main_navigation click_navigation_link [link_label] -
[groups:boards:show] toggle_swimlanes click_toggle_button - [is_active]
[projects:registry:index] registry_delete click_button - -
[projects:registry:index] registry_delete confirm_deletion - -
[projects:blob:show] congratulate_first_pipeline click_button [human_access] -
[projects:clusters:new] chart_options generate_link [chart_link] -
[projects:clusters:new] chart_options click_add_label_button [label_id] -

* If you choose to omit the category you can use the default.
** Use property for variable strings.

Reference SQL

Last 20 reply_comment_button events

FROM legacy.snowplow_structured_events_all
  event_label = 'reply_comment_button'
  AND event_action = 'click_button'
  -- AND event_category = 'projects:issues:show'
  -- AND event_value = 1
ORDER BY collector_tstamp DESC

Last 100 page view events

  -- page_url,
  -- page_title,
  -- referer_url,
  -- marketing_medium,
  -- marketing_source,
  -- marketing_campaign,
  -- browser_window_width,
  -- device_is_mobile
FROM legacy.snowplow_page_views_30
ORDER BY page_view_start DESC

Top 20 users who fired reply_comment_button in the last 30 days

  count(*) as hits,
FROM legacy.snowplow_gitlab_events_30
  se_label = 'reply_comment_button'
  AND gsc_pseudonymized_user_id IS NOT NULL
GROUP BY gsc_pseudonymized_user_id, se_category, se_action
ORDER BY count(*) DESC

Query JSON formatted data

  contexts:data[0]:data:extra:old_format as CURRENT_FORMAT,
  contexts:data[0]:data:extra:value as UPDATED_FORMAT
FROM legacy.snowplow_structured_events_all
WHERE event_action in ('wiki_format_updated')
ORDER BY derived_tstamp DESC

Web-specific parameters

Snowplow JavaScript adds web-specific parameters to all web events by default.

Snowplow monitoring

For different stages in the processing pipeline, there are several tools that monitor Snowplow events tracking:

  • Product Intelligence Grafana dashboard monitors backend events sent from instance to collectors fleet. This dashboard provides information about:
    • The number of events that successfully reach Snowplow collectors.
    • The number of events that failed to reach Snowplow collectors.
    • The number of backend events that were sent.
  • AWS CloudWatch dashboard monitors the state of the events processing pipeline. The pipeline starts from Snowplow collectors, through to enrichers and pseudonymization, and up to persistence on S3 bucket from which events are imported to Snowflake Data Warehouse. To view this dashboard AWS access is required, follow this instruction if you are interested in getting one.
  • SiSense dashboard provides information about the number of good and bad events imported into the Data Warehouse, in addition to the total number of imported Snowplow events.

For more information, see this video walk-through.

Related topics