A product feed is a file that contains your product catalog (all your products, with product metadata). It powers affinity-based personalization, product recommendations, social proof messaging, and more.
The following is the framework for creating and syncing your product feed with Dynamic Yield. The planning is critical from a business perspective because the feed columns determine what information you can use to personalize your campaigns. Work with your dev team to create the feed file, and to determine how and how often it should be synced.
You can create and sync product feeds using data files or via API.
Sync your product feed using a data file
- Create the file with all the required product metadata (for example, price or SKU).
- Sync the file with Experience OS by any of the listed methods.
- Parse the source file (optional). Use JavaScript code to manipulate the source file to adjust it to the format required by Experience OS (such as changing attribute names).
- Preview and save.
- Validate the sync status.
Check out our best practices for product feed design to get the most our of your product feed
Step 1: Create the file
For the technical details, see our Developer Docs.
Before you can upload and sync a product feed to Experience OS, make sure your feed file complies with the size, format, and content requirements.
| Feed Size | File Type |
| Up to 20,000 items | CSV, JSON, XML |
| 20,000-200,000 | CSV, JSON |
| More than 200,000 | CSV |
Note: If your feed includes more than 20,000 items, contact your Technical Account Manager to help with the sync (steps 2-4). The feed size is limited to 5 million items.
The file can be stored in any URL or a local folder, or in an Amazon S3 bucket that we provide. You can transfer the feeds to Amazon S3 using SFTP as well. If you use a CSV file, items must be separated by commas. See Step 2 for more information on these methods.
Product feed structure
The product feed must include the basic information about your product (SKU, group ID, price, image, URL, in stock or not, and its categories).
| Column (case sensitive) |
Description |
| sku | Product Unique Identifier. Note: SKUs cannot contain spaces or the string "//" |
|
product_type Optional but strongly recommended |
The type of item: Offer, Product, or Article Financial Institutions only. |
| group_id |
Identifies a group of products that differ in some product attributes. Note: If you change the group ID structure or category structure between syncs, the feed might reset, causing all previous recommendation data to be lost |
| name | Product Name |
| url | URL to the product details page (must be a valid URL, starting with HTTP/HTTPS) |
| price | Price of the product (must be a number) |
| in_stock | Indication whether the product is in stock (noted by ‘true/false’) |
| image_url | The URL to the product image (must be a valid URL, starting with HTTP/HTTPS) |
| categories | The categories associated with the product from general to specific |
|
keywords Optional but strongly recommended |
Any additional information describing the product, separated by pipes. Used for our machine learning and affinity algorithms. Because this column is optional, blank values will not trigger errors or warnings. |
If a product is missing a value in a mandatory column, it's skipped. If more than 10% of products have an empty value in one of the columns, the entire sync fails.
Custom columns
You can also add more columns that describe your product. These can later be used for targeting (for example, add "color", and target users who purchased blue items), affinity scores (for example, include color affinity in the affinity recommendation algorithm), or to define merchandising rules in your recommendation widgets (for example, never recommend products of a specific brand). See our Developer Docs for details.
Multi-language support
To support multiple languages, you can add a column for every translated version of a property you want to include. Learn more about multi-language support in your section.
Step 2: Create or sync the file with Experience OS
Go to Assets › Data Feeds and create a new product feed.
After naming the feed, you can select one of the following sync options:
- Sync a URL or upload a file: Upload a feed file (up to 25 MB), or refer to a URL location where a feed file is stored. If you are using HTTP authentication, add the username and password to the feed URL as follows http://user:password@url.
- Sync securely via Amazon S3
- Sync securely via SFTP
See our Developer Docs for detailed information.
Sync frequency
By default, the feed syncs with the source file every 24 hours. You can change this in Advanced Settings if your product catalog updates more frequently. If you sync the file via S3 or SFTP, Experience OS checks the bucket for a new file six times a day. If you've just uploaded your file to the bucket, you can click Sync Now to force the initial sync.
Step 3: Parse the source file (optional)
If the columns and values in your feed file don’t meet the requirements described in Step 1, or you want to merge files, or if some other manipulation is required to duplicate, move around, or rename values and fields, you can modify the feed using a parser function created in JavaScript. For example, XML files that are not "flat" can be "flattened" using a parser function.
Note: This is available for feeds that have fewer than 200,000 items and are smaller than 180 MB.
For the technical details, see our Developer Docs.
Step 4: Preview and save
If your feed source is an uploaded file or URL, you can view a preview of up to 100 entries of your feed, including any errors or warnings by clicking Preview. Click Save and Activate to proceed or Cancel to go back.
The synchronization duration depends on the number of items in your feed. For feeds with 250,000 items in the feed, it usually takes a few minutes, and 30 minutes for every additional 500,000 items in your feed. If you require a faster sync, consider updating your feed by API to update incremental changes instead of the standard full-feed update.
Note: For feeds with more than 2 million products, products with no associated events are filtered out of the data feeds preview. Active products are products that have one or more activities (product view/add-to-cart/purchase) in the last 6 months. So when you view the product feed in the preview, the number of products at the bottom of the page is the number updated with this filter.
Step 5: Validate sync status
You can verify that your feed is synchronizing successfully at any time:
- Go to Assets › Data Feeds.
- In the Last Sync Attempt column, you should see a Date Feed Synced icon
and a recent date. If the icon indicates an error or warning, download the log for more details.
- In the Actions column, click the View icon
. Validate that all required columns and any custom columns you have added exist and have the correct data.
Optional: Download the files locally via Terminal
See our Developer Docs for details.
Sync your product feed via API
If you have a large feed, you might choose to use an API to update only changes in your feed (remove or add an item, change item properties, and so on), for quicker updates.
This method requires developer know-how. Details are on our Developers site.
Syncing your product catalog via API with Dynamic Yield includes the following steps:
- Create the Product Feed in Experience OS. See the following section.
- Generate an API Key to include in each API call. See developer details.
- Implement the API calls to send data to Dynamic Yield. See developer details.
Create the product feed
- Go to Assets › Data Feeds and create a new product feed.
- For the Feed Source, select Sync via API.
- Add any optional properties that will be included in each item of the product feed by clicking New Property. These can later be used for targeting (for example, if you add "color", you can use it to target users who purchased blue items), affinity score (for example, include color affinity in the affinity recommendation algorithm), define merchandising rules in your recommendation widgets (for example, never recommend products of a specific brand).
- Custom property values must be strings, up to 1,000 characters, with no special characters.
- Custom property names must not start with '_id'.
- After you add a custom property, you can't remove it. You can stop using it or return empty values, but the property must be in any future items.
- To support multiple languages, you can add a property for every translated version of a property you want to include. Learn more about multi-language support in your section.
- Click Save and Activate to complete the process.