Realtime Data Streams

Build high-throughput, low-latency stream processing pipelines with advanced event pattern matching and backpressure management.

🚀 Get Started 🔧 API Docs

Core Concepts

Critical features of our stream processing architecture.

Backpressure Handling

Prevents system failures during sudden high-volume data surges with dynamic queue size adjustments.

Window Management

Optimized tumbling and sliding windows for time-based aggregations at scale.

Pattern Detection

Complex event pattern recognition with built-in stream filters for high signal-to-noise ratios.

Getting Started

1. Initialize Stream

Create a new stream processor with the eisen.stream() API.

    const pipeline = new eisen.Stream({
      backpressure: 'dynamic',
      throughputLimit: 10000,
      windowSize: '5s',
      bufferCapacity: 100000
    });

2. Connect Sources

Attach multiple real-time data sources using connectTo().

    pipeline.connectTo([
      {
        type: 'kafka',
        topic: 'iot-stream',
        consumers: 24,
        autoCommit: true
      },
      {
        type: 'websocket',
        url: 'wss://data-feed.example.com'
      }
    ]);

💡 Tip: You can monitor source health using the built-in dashboard

Advanced Operations

Powerful tools for complex stream processing needs

Session Windows

Intelligent session management with automatic user behavior tracking for session-based analytics.

Stateful Processing

Maintain persistent state across stream elements for pattern continuation tracking.

Implementation

stream-processor.eisen



import { Stream } from 'eiseniiiaaaaiia/streams';

// Initialize stream processor
const pipeline = new Stream({
  backpressure: 'elastic',
  throughputCap: 18000,
  checkpointInterval: '1s',
  state: { max: 1000000 }
});

// Define processing logic
pipeline.on('event', (data) => {
  if (data.type === 'click') {
    pipeline.metrics.increment('user-interactions');
    
    // Complex pattern matching
    if (pipeline.pattern({
      sequence: [
        { type: 'hover', within: '2s' },
        { type: 'click', within: '500ms' }
      ],
      limit: 1
    })) {
      pipeline.alert('Potential fraud pattern detected');
    }
  }
});

// Register sinks
pipeline.sink('console');
pipeline.sink('prometheus', {
  port: 9090
});

// Start processing
pipeline.start();

Copy