🏗️ Framework Architecture
This document provides a high-level overview of how the Ocean framework works. For detailed information about specific aspects, see the architecture sections below.
Overview
Ocean is a framework that orchestrates data synchronization between third-party systems and Port. It handles:
- Event Management: Listening for resync requests from Port
- Data Extraction: Running user-defined resync functions
- Data Transformation: Converting raw data to Port entities using JQ mappings
- State Synchronization: Comparing and syncing entities with Port
- Real-time Updates: Processing live events from third-party systems
Architecture Documentation
- Initialization Flow - How Ocean initializes when an integration starts
- Event Listeners - How event listeners monitor and trigger resyncs
- Data Flow - Complete data flow from resync trigger to Port synchronization
- Live Events Processing - How Ocean processes live events from third-party systems
- Framework Internals - Event context, metrics, error handling, and caching
High-Level Architecture
Summary
The Ocean framework orchestrates a complete ETL pipeline:
- Listen: Event listeners monitor for resync requests
- Extract: User code fetches data from third-party APIs
- Transform: JQ mappings convert raw data to Port entities
- Load: State applier syncs entities with Port
- Monitor: Metrics track the entire process
All of this happens automatically - you just need to implement the @ocean.on_resync() functions that yield raw data batches.