Riffle: Reactive Relational State for Local-First Applications

Document Title

Rifle: Reactive Relational State for Local-First Applications

Document Information

• Subject Area: Reactive data management combining user interface development and local-first architecture
• Keywords: Local-first software, UI state management, reactive programming, relational database, incremental view maintenance, synchronized transactional updates

Research Background and Problem

• Identified Problems or Challenges:

  • UI frameworks often compromise between simplicity and scalability.
  • Developers need to handle complex cross-layer state management (e.g., ORM, databases, caches, views, etc.).
  • State management in web applications often involves asynchronous operations, increasing complexity and potential inconsistencies.
  • Existing frameworks mostly provide partial support for reactive development and fail to cover the full process from UI to data layers.

• Significance:

  • Efficient state management can significantly enhance productivity in developing complex applications.
  • Incrementally complex state management is better suited to real-world application needs, especially in scenarios with large data volumes and high-performance requirements.

• Research Motivation and Related Work:

  • Drawing inspiration from the reactive data model of traditional spreadsheets while overcoming their performance and complexity limitations.
  • The rise of local-first architectures facilitates offline use and asynchronous synchronization.
  • Incremental view maintenance (IVM) in relational databases offers the potential for high-performance state updates.

Solution

• Proposed Method:

  1. Introduce a reactive relational architecture:
     • Data transformations are defined using relational queries with a DAG (Directed Acyclic Graph), providing a "declarative programming" interface.
     • UI and domain state are unified and stored in a local relational database, enabling transactional synchronized updates.
  2. Design and implement the Rifle system:
     • Built using a TypeScript library, integrating React.js and the SKDB database to support fast incremental updates and network synchronization.
  3. Implement a case study application: Overtone, a music management app, to explore Rifle's applicability in complex data and high-performance scenarios.

• Innovations in the Solution:

  • Unified reactive relational queries support synchronized updates between data and UI states.
  • The local-first model avoids inconsistencies caused by asynchronous operations, significantly improving user experience.
  • The introduction of declarative models like SQL allows developers to efficiently design data transformation logic, enhancing development intuitiveness.

• Implementation Steps and Techniques:

  • Store all application states in a locally persistent relational database (e.g., SKDB).
  • Use transaction-level reactive control for UI and business state updates, ensuring full synchronization.
  • Optimize update costs using incremental view maintenance techniques and incorporate a dynamic query generation mechanism.
  • For large datasets, combine custom GraphQL and SKDB's incremental update capabilities to provide high-performance support.

Research Outcomes

• Specific Results:

  • Developed the Overtone music management application using Rifle, successfully simplifying UI and data state management.
  • Overtone demonstrated fast UI responsiveness and flexible data view support, performing well even with a large number of tracks and complex relationships.
  • Achieved support for reactive write transactions, enabling more efficient and reliable offline-first operations.
  • Developers can write declarative queries with Rifle, simplifying and clarifying the handling of data dependencies.

• Comparative Advantages:

  • Compared to traditional web frameworks:
    • Eliminates inconsistencies and latency issues introduced by network boundaries.
    • Integrates UI state management with business-layer state management.
    • Avoids requiring developers to write cumbersome state propagation logic.
  • Compared to raw database support:
    • Enhances reactive capabilities in relational queries, optimizing performance.
    • Provides a complete development toolchain, particularly for UI and data integration scenarios.

• Experimental and Evaluation Results:

  • Performance tests show that using an incremental relational database (e.g., SKDB) can achieve data update times of <3ms for datasets with millions of tracks, whereas traditional SQLite requires hundreds of milliseconds for full view recomputation.
  • In the music management application, the UI exhibited near-instantaneous response (e.g., selecting playlists) even with tens of thousands of data entries, meeting the interaction standard of under 100ms.
  • Replacing single SQL with GraphQL provided greater expressiveness, allowing the UI to more easily generate nested data structures.

• Limitations and Future Directions:

  • Limitations:
    • Local-first software has limited applicability and is unsuitable for applications requiring frequent state updates, such as social networks or e-commerce platforms.
    • The current relational data model cannot efficiently represent complex hierarchies and sequences.
    • Unavoidable asynchronous operations (e.g., network requests, slow queries) still require additional logic from developers.
  • Future Improvements:
    • Design new declarative models (e.g., DIEL for handling network boundaries) to address more complex user scenarios.
    • Develop more efficient mechanisms for cross-device data synchronization and collaboration support.
    • Become part of a cross-application data interoperability platform, such as integration with customized analytics tools.

Conclusion

Rifle introduces a novel architecture that deeply integrates UI with fast reactive relational databases, addressing numerous pain points in current web application development through a local-first model. The Overtone case study validates the applicability of the Rifle system in high-complexity and high-performance scenarios, providing significant inspiration for future user application development.