This version is still in development and is not considered stable yet. For the latest stable version, please use StreamX Guides 1.0.0!

Java Ingestion Client

The StreamX Ingestion API Java Client allows developers to use the StreamX Ingestion API directly from their Java applications. This client enables to feed data into the StreamX by making HTTP requests by REST Ingestion Service.

How to add to the project

With Maven, add the following dependency to your pom.xml.

<dependency>
    <groupId>dev.streamx</groupId>
    <artifactId>ingestion-client</artifactId>
</dependency>

Usage

The Java Ingestion Client provides a StreamxClient as the main entry point for interacting with the REST Ingestion Service from a Java application. StreamxClient can be initialized with the restIngestionHost, which is the URL of the REST Ingestion Service.

StreamxClient client = StreamxClient.create(restIngestionHost);

To initialize a reusable publisher you must provide the 'channel' and associated data model class.

Publisher<Data> publisher = client.newPublisher(channel, Data.class);

The list of available channels and the schemas registered to StreamX can be found by querying the REST Ingestion Service /schema endpoint.

Publish data

Publish the data under specific dataKey.

publisher.publish(dataKey, data);

Unpublish data

Unpublish the data under specific dataKey.

publisher.unpublish(dataKey);

Data model class

The REST Ingestion Service accepts data in Avro format, so effective data management requires to define @AvroGenerated data model classes.

Generating data model classes from Apache Avro

AVRO provides a Maven plugin for generating data model classes from AVRO schema files.

<plugin>
    <groupId>org.apache.avro</groupId>
    <artifactId>avro-maven-plugin</artifactId>
</plugin>

Schema files could be retrieved from the REST Ingestion Service /schema endpoint.

Reusing data model classes

You can reuse data model classes from your StreamX mesh services implementations. To achieve that you can copy these classes to your project codebase.

Remember to maintain the same java packages structure consistent between the project and services code base. The package name is a part of AVRO schema, and it is validated.

Configuration

The Java Ingestion Client offers several configuration options to meet specific needs.

Custom HTTP requester configuration

HttpRequester is an interface that allows for injecting a custom HTTP client implementation. If this is set, the custom Apache HTTP Client Configuration is ignored.

HttpRequester customHttpRequester = new CustomHttpRequester();
StreamxClientBuilder streamxClientBuilder = StreamxClient.builder(restIngestionHost)
    .setHttpRequester(customHttpRequester);
StreamxClient client = streamxClientBuilder.build();

Custom Apache HTTP Client configuration

The default HttpRequester implementation uses Apache HTTP client. You can change the HTTP client by using StreamxClientBuilder. Default client is used when not provided.

CloseableHttpClient httpClient = HttpClients.custom().build();
StreamxClientBuilder streamxClientBuilder = StreamxClient.builder(restIngestionHost)
    .setApacheHttpClient(httpClient);
StreamxClient client = streamxClientBuilder.build();

Authentication configuration

REST Ingestion Service can be secured with JWT Token. In that case authentication token must be passed to the StreamXClient.

StreamxClientBuilder streamxClientBuilder = StreamxClient.builder(restIngestionHost)
    .setAuthToken(token);
StreamxClient client = streamxClientBuilder.build();