Serving dynamic content with StreamX and AEM

In this tutorial, you will learn how to create a Dynamic Carousel Component that displays data coming from StreamX.

You will use Apache Dispatcher with the SSI module enabled to make the Carousel Component dynamic, allowing it to display frequently changing data without the need for cache invalidation.

This tutorial covers the following topics:

  • AEM and Dispatcher setup using AEM Compose

  • Running the StreamX Mesh

  • Building new Dynamic Carousel Component on AEM

  • Feeding the StreamX Mesh with data using StreamX CLI

Prerequisites

Make sure that no other application uses ports 80, 4502 and 4503 (AEM ports) and 8081 (StreamX REST Ingestion port)

Step 1: Get the sources

Clone the Git repository containing source files for the example:

git clone https://github.com/streamx-dev/streamx-docs-resources.git

Step 2: Set up AEM and Dispatcher

  1. Open the terminal and navigate to serving-dynamic-content-aem-tutorial inside the cloned project directory.

  2. Copy your AEM 6.5 installation file and license file to the [cloned-repository-path]/serving-dynamic-content-aem-tutorial/aem/home/lib directory.

  3. Set up your AEM stack by executing the following command:

    sh taskw setup
    Be patient, the initial setup may take a long time as it creates author, publish and dispatcher instances. Once it’s done you can verify the setup by visiting http://publish.aem.local/, where you should see the We.Retail landing page.

    setup command needs to be run only once. When you finish you can stop AEM stack and start again:

    sh taskw stop
sh taskw start

Step 3: Run the StreamX Mesh

  1. Open the terminal and navigate to serving-dynamic-content-aem-tutorial inside the cloned project directory.

  2. Run the StreamX Mesh using command:

    streamx run

  3. Wait for the following output:

    -------------------------------------------------------------------
STREAMX IS READY!
-------------------------------------------------------------------
...
-------------------------------------------------------------------
Network ID:
...
Mesh configuration file: ./mesh.yml
-------------------------------------------------------------------

Step 4: Feed the StreamX Mesh

You need to inform the StreamX Mesh how to generate the content for the new component.

Remember to run streamx commands from the same folder, as they rely on relative paths.

  1. Let’s publish the renderer of the component which has HTML content with some placeholders to be filled by StreamX when data arrives.

    streamx publish -s 'template.bytes=file://templates/carousel.html' renderers carousel.html

  2. Publish the context with all required information about what and how it should be generated.

    streamx publish -j 'file://context/fragments-rendering-context.json' rendering-contexts fragments-rendering-context

  3. Add some sample product data to StreamX by executing the following 3 commands:

    streamx publish -s 'content.bytes=file://products/product_1.json' data product:1
    streamx publish -s 'content.bytes=file://products/product_2.json' data product:2
    streamx publish -s 'content.bytes=file://products/product_3.json' data product:3

  4. Visit http://localhost:8081/published/weretail/_fragments/collected:products:cheapest-by-category:Biking.carousel.html in any web browser, and make sure three published product are present, though unstyled.

Let’s break down the URL:

  • localhost:8081 is where the StreamX Web Delivery Service listens for requests.

  • /published/weretail/_fragments/ is the base prefix for carousel fragments.

  • collected:products:cheapest-by-category:Biking is referring to an aggregated dataset collected:products:cheapest-by-category (cheapest products grouped by category) with Biking (category) as a key.

  • .carousel.html is the suffix for carousel fragments.

Step 5: Create Dynamic Carousel Component

  1. Create the component

    • Log in to AEM author with default credentials admin/admin.

    • Visit AEM author - CRXDE - /apps/weretail/components/content.

    • Right-click on the content node and select Create ... / Create Component.

    • Fill in the following properties:

      • Label: dynamicproductcarousel

      • Title: Dynamic Product Carousel

      • Group: We.Retail

    • Save your changes.

  2. Replace component’s template

    • Rename dynamicproductcarousel/dynamicproductcarousel.jsp to dynamicproductcarousel/dynamicproductcarousel.html and save the changes.

    • Replace node’s content with the following snippet:

      <sly data-sly-test="${wcmmode.disabled}">
    <!--#include virtual="/published/weretail/_fragments/collected:products:cheapest-by-category:Biking.carousel.html" -->
</sly>
<sly data-sly-test="${!wcmmode.disabled}">
    Product carousel's content will be injected by Dispatcher through SSI.
</sly>

    • Right-click on the newly created dynamicproductcarousel node and select Create ... / Create Dialog with just the defaults.

    • Save your changes again.

  3. Activate the component

    • Visit AEM author - Activate Tree page.

    • Use /apps/weretail/components/content/dynamicproductcarousel as start path.

    • Untick Only Modified and do the activation.

Step 6: Add the component to the page

  1. Visit the AEM author - EN Blueprint page.

  2. Add a new Dynamic Product Carousel component above the Title component with text DISCOVER THE FINEST GEAR.

  3. Use the Rollout Page option from the page action bar.

  4. Visit AEM author - US EN page.

  5. Use the Publish Page option from the page action bar.

  6. Visit http://publish.aem.local/content/we-retail/us/en.html in any browser (it may take some time for AEM to compile all the client libraries if you are visiting a We.Retail page for the first time), and you should see a fully functional carousel with 3 products above the DISCOVER THE FINEST GEAR text, as shown on the image below.

    image
    Figure 1. Dynamic Carousel

If you take a closer look at the order of the items within the carousel, you will notice that they are ordered by price in ascending order, exactly the way our StreamX Mesh was configured.

Step 7: Update dynamic content

  1. Add the fourth product (with the lowest price) by executing:

    streamx publish -s 'content.bytes=file://products/product_4.json' data product:4

  2. Visit http://publish.aem.local/content/we-retail/us/en.html again. Make sure that:

    • The new product is located at the first place.

    • Other products are moved to the right (so the previous 3rd product is no longer available in the carousel, due to the StreamX Mesh configuration).

Summary

Congratulations! You have just built and validated a new AEM component which is able to dynamically load product information from StreamX.

Related content

Dispatcher configuration

  • Allow SSI in conf.d/enabled_vhosts/we-retail_publish.vhost

    Options +Includes
AddOutputFilter INCLUDES .html

  • Prevent product carousel fragment URLs from being rewritten in conf.d/rewrites/we-retail_rewrite.rules

    RewriteCond %{REQUEST_URI} !^/published/weretail/_fragments

  • Allow proxying requests for product carousel fragments toward StreamX in conf.d/proxy/streamx.proxy

    ProxyPassMatch "^/(.*/_fragments/.*\.carousel\.html)$" "http://host.docker.internal:8081/$1"