Flink Kinesis Connector, Apache Flink provides connectors for reading from files, sockets, collections, and custom sources.
- Flink Kinesis Connector, There are two available Table API and SQL distributions for the Kinesis connector. Apache Flink provides connectors for reading from files, sockets, collections, and custom sources. Aug 3, 2025 · Amazon Kinesis Data Streams Connector The Kinesis connector allows users to read from and write to Amazon Kinesis Data Streams. For more information on writing to Kinesis Data Streams from Apache Flink, see Amazon Kinesis Data Streams Connector. The examples demonstrate how to use Kinesis as both a source and sink for Flink streaming applications, including standard polling mode and Enhanced Fan-Out (EFO) consumer options. It covers the architecture, usage patterns, and configuration options for the following Kinesis-related connectors: Apache Flink is a framework and distributed processing engine for stateful computations over unbounded and bounded data streams. kinesis</groupId> <artifactId>amazon-kinesis-connector-flink</artifactId> May 16, 2025 · This document provides a technical overview of the Apache Flink connectors for Amazon Kinesis services. What is the problem this project solves? This library allows you to migrate from the Amazon to Apache Kinesis connector while retaining state in the source operator. Nov 22, 2024 · Amazon Managed Service for Apache Flink now offers a new Apache Flink connector for Amazon Kinesis Data Streams. x. It is sometimes desirable to have Flink operate as a source or sink against a Kinesis VPC endpoint or a non-AWS Kinesis endpoint such as Kinesalite; this is especially useful when performing functional testing of a Flink application. Linking to the flink-connector-kinesis will include ASL licensed code into your application. Note To use the Kinesis connector for the following application, you must download the source code for the connector and build it as described in the Apache Flink documentation. The Kinesis connector is not part of the binary distribution. The flink-connector-kinesis_2. 0, introduces a new source connector to read data from Amazon Kinesis Data Streams. Oct 31, 2020 · Both features are already available in the official Apache Flink connector for Flink 1. For an example of an application that uses a Kinesis data stream for input and output, see Tutorial: Get started using the DataStream API in Managed Service for Apache Flink. 10 has a dependency on code licensed under the Amazon Software License (ASL). x onwards, only KinesisStreamsSource and DynamoDbStreamsSource, which use the new Source interface will be supported. Jun 20, 2026 · Tags aws amazon streaming flink apache connector kinesis connection Ranking #36411 in MvnRepository HomePage https://flink. Connectors consist of complete modules for interacting with Amazon services and third-party systems. 0 and provides several enhancements. From Flink 2. 0 There is no connector (yet) available for Flink version 2. This open-source connector, contributed by AWS, supports Apache Flink 2. - awslabs/amazon-kinesis-connector-flink Use Kinesis data streams Apache Flink provides information about the Kinesis Data Streams Connector in the Apache Flink documentation. It enables in-order reads during stream scale-up or scale-down, supports Apache Flink's native watermarking, and improves observability through unified connector metrics May 18, 2025 · This document describes the implementation of Apache Flink connectors for Amazon Kinesis Data Streams in the Amazon Managed Service for Apache Flink examples repository. Connectors are flexible integrations that let you read from files and directories. org 🔍 Inspect URL Links Amazon Managed Service for Apache Flink Examples Example applications in Java, Python, Scala and SQL for Amazon Managed Service for Apache Flink (formerly known as Amazon Kinesis Data Analytics), illustrating various aspects of Apache Flink applications, and simple "getting started" base projects. Types of connectors include the Nov 25, 2024 · Why did we need new source connectors? We implemented new source connectors because the FlinkKinesisConsumer and FlinkDynamoDBStreamsConsumer use the deprecated SourceFunction interface, which is removed in Flink 2. Use Kinesis data streams The flink-connector-kinesis_2. See how to link with it for cluster execution here. In your application code, you use an Apache Flink source to receive data from a stream. <groupId>software. Dependency To use this connector, add the below dependency to your project: There is no connector (yet) available for Flink version 2. This is a fork of the Apache Flink Kinesis connector adding Enhanced Fanout support for Flink 1. In the Amazon Managed Service for Apache Flink DataStream API, connectors are software components that move data into and out of a Managed Service for Apache Flink application. amazon. Dec 18, 2024 · This new release, version 5. 8/1. 10 artifact is not deployed to Maven central as part of Flink releases because of the licensing issue. Add the following dependency to your project to start using the connector. 12. Jan 12, 2026 · Flink : Connectors : AWS : Amazon Kinesis Data Streams Flink : Connectors : AWS : Amazon Kinesis Data Streams Overview Versions (185) Used By (10) Badges Books (16) License Apache 2. For use in PyFlink jobs, use the following dependency: In order to use the in PyFlink jobs, the following dependencies are required:. You no longer need to build the Kinesis Connector from source. What is the Apache Kinesis Connector? The Apache Flink Kinesis Connector (see on GitHub) is supported by the Apache Flink community and the recommended Amazon Kinesis connector. 0. 3. 1. apache. Amazon Kinesis Flink Connectors This repository contains various Apache Flink connectors to connect to AWS Kinesis data sources and sinks. This section describes the sources that are available for Amazon services. In this post, we explain how the new features of this connector can improve performance and reliability of your Apache Flink application. 11 on KDA. yagu, jnc24, fwgr, w3, jzap, 83o, e17a, 2pu, 5ip, 291j5,