Apache Flink
Distributed stream and batch data processing engine
Activity unknown
No repository statistics available for this project.
Apache Flink is a distributed processing engine for stateful stream processing and batch analytics. It is designed for low-latency, high-throughput pipelines with strong consistency, fault tolerance, and event-time processing.
Key Features
- Stateful stream processing with exactly-once consistency (depending on connector and sink support)
- Event-time semantics with watermarks and advanced windowing
- Fault tolerance via checkpoints and savepoints for upgrades, rollbacks, and migrations
- Unified runtime for streaming and batch workloads
- Rich APIs including DataStream and Table/SQL for declarative processing
- Scalable parallel execution on clusters with fine-grained state management
Use Cases
- Real-time analytics and monitoring pipelines over logs and events
- Stream ETL and enrichment between messaging systems and databases
- Stateful event-driven applications such as fraud detection or alerting
Limitations and Considerations
- Operating Flink reliably requires careful tuning of state backends, checkpoints, and connector configuration
- Some delivery guarantees depend on the chosen connectors and sinks, not only the core engine
Apache Flink is well-suited for teams building reliable, stateful real-time systems and unified streaming/batch data pipelines. It provides robust primitives for event-time processing and recovery, while scaling from small deployments to large cluster environments.
Categories:
Tags:
Tech Stack:
Similar to Apache Flink
Vector
High-performance observability data pipeline written in Rust
Open-source observability pipeline to collect, transform, and route logs and metrics with a single, high-performance binary and programmable transforms.
Quickwit
Cloud-native, sub-second search on cloud storage for logs and traces.
Open-source cloud-native search engine for observability data on object storage with an Elasticsearch/OpenSearch-compatible API.
Janus WebRTC Server
General-purpose WebRTC media server with a plugin architecture
Janus WebRTC Server is a lightweight, general-purpose WebRTC gateway for building real-time audio/video applications via plugins and a JSON-based API.
OpenTripPlanner
Multimodal trip planning and transit routing server
OpenTripPlanner (OTP) is an open source multimodal routing engine that builds networks from GTFS and OpenStreetMap to produce itineraries and real-time transit trip plans via APIs.
Tracearr
Real-time monitoring and analytics for Plex, Jellyfin, and Emby
Real-time monitoring platform for Plex, Jellyfin, and Emby with session tracking, playback analytics, stream maps, alerts, and account-sharing detection rules.
uStreamer
Lightweight MJPEG-over-HTTP video streaming server for V4L2 devices
uStreamer is a lightweight, fast MJPEG-over-HTTP streamer for V4L2 capture devices, optimized for high-FPS HDMI/VGA capture and PiKVM-style KVM over IP use.