This version is still in development and is not considered stable yet. For the latest stable version, please use StreamX Guides 1.0.0!

Set up search with StreamX and AEM

In this tutorial we will set up StreamX search with AEM used for data source. We will use the We.Retail sample application and content which comes pre-installed with non-production AEM author installations.

Prerequisites

To complete this tutorial, you will need:

  • Roughly 15 minutes.

  • StreamX CLI installed.

  • Git installed.

  • jq installed.

  • A running instance of AEM author 6.5 instance with at least Service Pack 6.5.17 installed, and also with the out-of-the-box We.Retail sample application and content.

If you have an author instance that wasn’t started with the nosamplecontent run mode, you can safely assume that you have this installed. You can validate the installation of We.Retail by visiting We.Retail landing page in the English master. If it returns a 404, then you don’t have a proper We.Retail installation. In this case, please use another (for example a new & fresh) AEM author instance.

Step 1: Get the source files

Clone the Git repository containing source files for the example:

git clone https://github.com/streamx-dev/streamx-docs-resources.git

Step 2: Install StreamX OSGi bundles and configuration

To integrate AEM with StreamX, you must install the StreamX OSGi bundle along with all the necessary OSGi dependencies and configurations. Once installed and configured, the OSGi bundle enables feeding StreamX Mesh with data sourced from AEM.

Follow the steps below to install the package:

  1. Visit AEM author - CRX Package Manager.

  2. Upload and install aem-with-streamx-tutorials/streamx-aem.all-1.0.2.zip from the cloned project repository.

Step 3: Run the StreamX Mesh

  1. Open the terminal and go to search-with-streamx-and-aem-tutorial inside the cloned project directory.

  2. Run the StreamX Mesh by using the following command:

    streamx run

  3. Wait for the following output:

    -------------------------------------------------------------------
STREAMX IS READY!
-------------------------------------------------------------------
...
-------------------------------------------------------------------
Network ID:
...
Mesh configuration file: ./mesh.yaml
-------------------------------------------------------------------

Step 4: Publish content from AEM

With the StreamX Connector OSGi bundles and configurations installed, you can now push content into the StreamX Service Mesh. Content ingestion is straightforward and involves performing the familiar activation, publication, or replication process. During replication events, the StreamX Connector automatically collects and sends the content to the StreamX Service Mesh, ensuring seamless integration.

  1. Visit AEM author - Sites admin page - We.Retail United States

  2. Select page /content/we-retail/us and from the top menu, click on Manage Publication.

  3. On the next screen (Options):

    • Leave the defaults unchanged.

    • Click Next.

  4. Set the following options:

    • Action : Publish.

    • Scheduling : Now.

  5. On the next screen (Scope):

    • Click the thumbnail of the /content/we-retail/us item to reveal the Include Children option.

    • Click the Include Children item and deselect every checkbox.

    • Confirm your changes by clicking on Add.

  6. Finally, click Publish

  7. Wait for AEM author to complete the publication process.

In the background, AEM sends the published content to the StreamX Service Mesh, which feeds data to the search service.

Step 5: Search content with StreamX

In this step, you will search the content of the We.Retail pages by using the StreamX Service Mesh. The goal is to query the search service for a specific term- equipment, in this example- and validate the results by counting the number of matches. This will confirm that the service is functioning correctly and is properly connected to the ingested content.

The search service is defined and running within the StreamX Service Mesh. It listens for requests at http://localhost:8082/search.

  1. To search for equipment and limit the results to 40 items, execute the following command in the terminal:

    curl -s 'http://localhost:8082/search/query?size=40&query=equipment' -H 'Accept: */*' --insecure | jq '.hits.hits | length'

    This command sends a search query to the service and uses jq to extract the count of matching items from the incoming JSON response.

  2. Now check the search output: it should display the number 40, indicating that at least 40 matches were found across all We.Retail pages.

Step 6: Update content

In this step, you will update content in AEM and observe how the changes are reflected in the search service.

  1. Verify the current search results:

    • First, confirm that there are no matches for the term StreamX. Run the following command in the terminal:

      curl -s 'http://localhost:8082/search/query?size=40&query=streamx' -H 'Accept: */*' --insecure | jq '.hits.hits | length'

    • Ensure the output is 0, indicating that there are no existing matches for the term.

  2. Update content in AEM:

    • Visit AEM author - US EN Equipment Blueprint page

    • Open the edit dialog of the Title component, located under the hero image. Currently it says Welcome our finest equipment.

    • Replace Welcome our finest equipment with Hello StreamX from AEM!

    • Confirm your changes in the edit dialog.

  3. Roll out the updated content:

    • Use the Rollout Page option from the page action bar to roll out the changes to the two existing live copies.

  4. Publish the updated content:

  5. Verify the updated search results:

    • Execute the search again, to confirm that the updated content has been indexed:

      curl -s 'http://localhost:8082/search/query?size=40&query=streamx' -H 'Accept: */*' --insecure | jq '.hits.hits | length'

    • The output should now display 1, indicating that the updated content was successfully found by the StreamX search service.

Summary

Congratulations! You have set up the StreamX search with AEM used for the data source.

Related content