If you started using the Daily Activity Stream before February 28, 2022, you will need to make the following changes by April 14, 2022:
- The Parquet files are now located at a new path, formatted like this:
- If you're using the US (COM) data center:
s3://dy-raw-data-export/sectionId=1234567
(instead of s3://dy-raw-data-export/1234567) - If you're using the EU data center:
s3://dy-raw-data-export-eu/sectionId=1234567
(instead of s3://dy-raw-data-export-eu/1234567)
- If you're using the US (COM) data center:
- This change applies also to the date subfolder, which is now formatted like this:
/date=2022-01-01 (instead of /2022-01-01). - The section ID attribute is no longer available within the data and needs to be extracted from the path you are connecting to.
- No need to generate new credentials in the Dynamic Yield console. Continue using the credentials you have.
You can export the data that Dynamic Yield tracks, and ingest it into your BI platform. The export is done on a daily basis, in a Parquet file that is exported to a secured S3 bucket. The data in the S3 bucket is stored for 30 days due to privacy regulations.
This data allows your BI team to:
- Build a customized report that is tailored to your unique business needs.
- Connect this data to additional data sources and build reports. For example, join CRM data and acquisition performance and build more holistic reports.
Data format
The file is in Apache Parquet format that is optimized for large data sets. The file name will contain the date, and the file content will include all of your traffic activities (one entry per activity).
There are 3 types of activities, with a different eventType attribute value, and different attributes according to the type :
| eventType | Description | Additional Attributes (Examples) |
|---|---|---|
| UIA | Pageview | URL, page context |
| DPX | Event hit | Event properties, event value |
| VARIATION_ENGAGEMENT | Variation impression or click | Campaign name, variation name |
These are the full list of attributes according to the eventType:
Represents a pageview that the user has made.
| Attribute | Description | Example |
|---|---|---|
| eventType String |
The type of activity. For pageview - value is UIA (as opposed to VARIATION_ENGAGEMENT or DPX) |
"UIA" |
| contextType String |
The page type, according to the page context. | "HOMEPAGE" |
| contextData String array |
The data about the page type.
|
["Women","Shoes"] |
| dyId Long |
The internal identifier Dynamic Yield assigns to each visitor in the site or app, unique per device. | 1.23456789098765E+018 |
| timestamp Long |
The time of the activity has occurred, in milliseconds from the UNIX epoch. | 1621798861400 |
| eventType String |
The type of event fired. It is always one of the following: DPX: An event using one of Dynamic Yield’s event APIs UIA: A pageview VARIATION_ENGAGEMENT: The engagement with a Dynamic Yield Variation UNIT_IMPRESSION: Check UNIT_CLICK: Check |
"DPX" |
| sessionId Integer |
The Internal identifier Dynamic Yield assigns to a visitor's session | 1234567890 |
| url String |
The URL from which the event was fired. | "https://www.example.com/?url_params=123" |
| urlClean String |
The URL from which the event was fired, after removing any URL parameters. | "https://www.example.com/" |
| audiences Integer array |
The list of audience identifiers the user is a member of at the time of firing the event. | [1234567, 9876543] |
| browser String |
The browser type from which the event was fired. | "Safari" |
| device String |
The type of device which triggered the event | "Tablet" |
| operatingSystem String |
The operation system of the device that fired the event. | "Mac OS X" |
| screenResolution String |
The screen resolution of the device that fired the event. | "Low (1024px and below)" |
| reqTimestamp Long |
Internal request timestamp to the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| procTimestamp Long |
Internal processing timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| resTimestamp Long |
Internal resolution timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
Represents a Dynamic Yield event that the user has fired.
| Attribute | Description | Example |
|---|---|---|
| eventType String |
The type of activity. For event hit - value is DPX (as opposed to VARIATION_ENGAGEMENT or UIA) |
"DPX" |
| eventId Integer |
A unique identifier for each event explicitly fired from the site. | 12345 |
| eventName String |
The event name as written in the event API. | "Purchase" |
| eventProperties String (JSON) |
The event properties as written in the event API. These differ depending on the eventType value. | { "transaction_id": "ABC123456", "value": 100.0, "currency": "USD", "dyType": "purchase-v1", "Brands": "Nike", "Categories": "Sneakers", "Number_of_items": 1.0, "cart": [{ "productId": "AIR-123", "quantity": 1.0, "itemPrice": 100.0 }] } |
| eventValue Long |
The monetary value in cents of an event as written in the event API. | 1000 |
| uniqueTransactionId String |
A unique transaction id for purchase events, as written in the purchase event API. | "ABC123456" |
| dyId Long |
The internal identifier Dynamic Yield assigns to each visitor in the site or app, unique per device. | 1.23456789098765E+018 |
| timestamp Long |
The time of the activity has occurred, in milliseconds from the UNIX epoch. | 1621798861400 |
| sessionId Integer |
The Internal identifier Dynamic Yield assigns to a visitor's session | 1234567890 |
| url String |
The URL from which the event was fired. | "https://www.example.com/?url_params=123" |
| urlClean String |
The URL from which the event was fired, after removing any URL parameters. | "https://www.example.com/" |
| audiences Integer array |
The list of audience identifiers the user is a member of at the time of firing the event. | [1234567, 9876543] |
| browser String |
The browser type from which the event was fired. | "Safari" |
| device String |
The type of device which triggered the event | "Tablet" |
| operatingSystem String |
The operation system of the device that fired the event. | "Mac OS X" |
| screenResolution String |
The screen resolution of the device that fired the event. | "Low (1024px and below)" |
| reqTimestamp Long |
Internal request timestamp to the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| procTimestamp Long |
Internal processing timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| resTimestamp Long |
Internal resolution timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
Represents an impression of a variation or a click on a variation.
| Attribute | Description | Example |
|---|---|---|
| eventType String |
The type of activity. For variation click or impression, the value is VARIATION_ENGAGEMENT (as opposed to UIA or DPX) |
"VARIATION_ENGAGEMENT" |
| engagementType String |
The type of engagement with the variation. Possible values:
|
"IMPRESSION" |
| campaignId Integer |
The ID of the campaign that this variation is part of. | 123456 |
| campaignName String |
The name of the campaign that this variation is part of. | "Homepage Banner" |
| experienceId Integer |
The ID of the experience that this variation is part of. | 123456 |
| experienceName String |
The name of the experience that this variation is part of. | "Summer Promo" |
| experimentId Integer |
The unique identifier of the test version. | 123456 |
| versionId Integer |
The unique identifier of the test version. An A/B test might have multiple versions. | 245467 |
| variationIds Integer array |
The ID of the variation that the user was served with (if the type is IMPRESSION) or clicked on (if the type is CLICK). In most cases, the list contains a single ID, but if the campaign type is "Dynamic Content Item List", it will contain a list of variations, separated with a comma. |
[1234567, 9876543] |
| variationNames String array |
The name of the variation that the user was served (if the type is IMPRESSION) or clicked on (if the type is CLICK). Usually, the list contains a single ID, but if the campaign type is "Dynamic Content Item List", it contains a list of variations, separated by commas. |
["Blue Button", "Red Button"] |
| sku String array |
If the event is viewing or clicking a recommendation widget, it's the list of SKUs that were recommended, or the SKU that was clicked. |
["1234", "9876"] |
| strategyId Integer array |
If the event is viewing or clicking a recommendation widget, it's the ID of the Strategy that was served. A single variation can include multiple widgets with multiple strategies. |
[126651,426356] |
|
strategyName |
If the event is viewing or clicking a recommendation widget, it's the name of the Strategy that was served. A single variation can include multiple widgets with multiple strategies. |
["Most Popular","Affinity"] |
| touchpointId Integer |
In touchpoints only: The ID of the touchpoint. | 245467 |
|
touchpointName |
In touchpoints only: The name of the touchpoint. | ["Hero Banner"] |
| parentVariationId Integer |
In touchpoints only: The ID of the variation that serves this touchpoint in the multi-touch campaign. | [9876543] |
|
parentVariationName |
In touchpoints only: The name of the variation that serves this touchpoint in the multi-touch campaign. | "Blue Design" |
| dyId Long |
The internal identifier Dynamic Yield assigns to each visitor in the site or app, unique per device. | 1.23456789098765E+018 |
| timestamp Long |
The time of the activity has occurred, in milliseconds from the UNIX epoch. | 1621798861400 |
| sessionId Integer |
The Internal identifier Dynamic Yield assigns to a visitor's session | 1234567890 |
| url String |
The URL from which the event was fired. | "https://www.example.com/?url_params=123" |
| urlClean String |
The URL from which the event was fired, after removing any URL parameters. | "https://www.example.com/" |
| audiences Integer array |
The list of audience identifiers the user is a member of at the time of firing the event. | [1234567, 9876543] |
| reqTimestamp Long |
Internal request timestamp to the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| procTimestamp Long |
Internal processing timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
| resTimestamp Long |
Internal resolution timestamp of the analytics pipeline, in milliseconds from the UNIX epoch. | 1621798861400 |
Turning on the Daily Activity Stream
To turn on the Daily Activity Stream:
- Go to Settings › General Settings › Daily Activity Stream.
- Click Turn on daily export.
- Copy the S3 bucket path and credentials to a secure location as it is only displayed once.
The path you are given to the S3 bucket looks like this:
s3://dy-raw-data-export/sectionId=1234567 or s3://dy-raw-data-export-eu/sectionId=1234567At this location, there are subfolders for each date, containing the Parquet files. They are formatted like this: /date=2022-01-01
Lost your credentials?
Click the additional options icon
and then click Generate New Credentials. Keep in mind that you can only generate new credentials once. - That's it! The export is prepared on a daily basis, exported to the S3 buckets during the night.
After the Daily Activity Stream is turned on, you can see the status of the export.
You can always disable the export in the options menu. If you disable the export, and then enable it again, your S3 bucket stays the same, but the credentials will change.
Importing data into your analytics platform
In order to analyze data based on Daily Activity Steam, you need to ingest the Parquet file from the S3 folder into the data warehouse used by your analytics platform. The documentation of your data warehouse should have any required information. Here are some links to the relevant articles of some common ones: