2025 Technical Kick-off

TL;DR

In 2025, we aim to enhance the platform's maintainability, scalability, and performance. Key topics include:

• Redesigning the file management system
• Addressing critical technical debt
• Developing a new CLI for the yuba-monorepo project
• Refactoring legacy components, such as the app's detail view
• Conducting POCs, including splitting the items table into dedicated SQL tables for each business table.

Introduction

This session aimed to present and prioritize key technical topics for 2025. It also served to raise awareness among the team about identified challenges.

Topics covered

The presentation used for discussion is available here: Technical Roadmap 2025.pptx

Subject 0: Redesigning File Management within the platform

Summary

Redesign and standardize file management across the Product-Live platform (app.product-live.com, settings.product-live.com, Data Factory). This topic was carried over from 2024 due to resource constraints.

Expected Benefits: Simplified architecture, improved file lifecycle management, reduced storage volume, better accessibility control, and enhanced usage tracking. This prepares the platform for future challenges, including increased data volumes and new file management features.

GitHub Issue: 📚 Rethinking File Management

Current Situation

• Three file types exist (items, job definitions, Data Factory-generated files).
• No clear distinction between file types.
• File lifecycle is unmanaged (e.g., deleting an item doesn't delete associated files).
• Limited accessibility controls and sharing options.
• Minimal usage metrics (e.g., volume exchanged, downloads).

Objectives

• Define a coherent file management architecture.
• Manage file lifecycles, especially temporary files from Data Factory.
• Track usage metrics for each account.

Subject 1: Account Creation Tool

Summary

Develop a tool for bulk creation and editing of resources (accounts, users, tables, etc.).

Expected Benefits: Streamlined account setup, reduced errors, standardized processes, and faster bulk updates.

GitHub Issue: 📚 Account Creation Tool

Current Situation

• Account setup involves numerous repetitive steps.
• These steps are repeated across development cycles (ticket qualification, QA, etc.).
• Other teams (support, service, pre-sales) face similar challenges.

Objectives

• Simplify account creation.
• Standardize account formats across teams.

Subject 2: Resolving Technical Debt

Summary

Complete the yuba logs project and Jiro migration.

Expected Benefits: Reduced technical debt, improved maintainability, and scalability.

GitHub Issue: 📚 Resolving Technical Debt

Current Situation

• Yuba logs are partially deployed.
• Jiro migration has unresolved issues.

Objectives

• Finalize yuba logs deployment.
• Complete Jiro migration.

Subject 3: Improved Yuba CLI

Summary

Rebuild the yuba-monorepo CLI for better maintainability and team knowledge sharing.

Expected Benefits: Enhanced team productivity and a more maintainable tool.

GitHub Issue: 📚 Yuba CLI

Current Situation

• The current CLI is outdated and poorly documented.

Objectives

• Build a new CLI using the current tech stack.
• Provide comprehensive documentation.

Subject 4: Unifying Conditional Formatting, Mapping, and Formula

Summary

Merge these closely related projects into a unified system.

Expected Benefits: Simplified codebase and easier feature additions.

Current Situation

• These projects are similar but not unified, leading to redundant development efforts.

Objectives

• Unify the projects under the "formula" functionality while maintaining backward compatibility.

Subject 5: Splitting the Items Table

Summary

Explore splitting the "items" table into dedicated SQL tables for each business table.

Expected Benefits: Support for larger item volumes and future scalability.

Current Situation

• A single SQL table holds all items.

Objectives

• Conduct a POC to assess feasibility.

Subject 6: Cleaning Up Feature Flags

Summary

Remove obsolete feature flags from the codebase and configcat tool.

Expected Benefits: Simplified code and improved maintainability.

GitHub Issue: 📚 Clean Up Feature Flags

Current Situation

• Many unused feature flags remain in the codebase.

Objectives

• Remove all unused feature flags.

Subject 7: Accessible Product Usage Analytics

Summary

Enable non-technical employees to access and analyze product usage data.

Expected Benefits: Data-driven decision-making and improved user experience.

Current Situation

• Data is not easily accessible to non-technical users.

Objectives

• Make data accessible, self-explanatory, and filterable.

Subject 8: Removing Haku

Summary

Replace Haku and Kanoashi with Calcifer.

Expected Benefits: Simplified architecture and improved maintainability.

Current Situation

• The haku API was made for our own use, and suffers from a patchwork effect making it harder to maintain and unusable outside out team ​

Objectives

• Transition to Calcifer.

Subject 9: Detail View Refactor

Summary

Refactor the detail view for better maintainability and scalability.

Expected Benefits: Improved user experience and maintainability.

GitHub Issue: 📚 Detail View Refactor

Current Situation

• Based on legacy React, lacks virtualization, and is not tested in Storybook.

Objectives

• Refactor from scratch, add virtualization, and include Storybook tests.

Subject 10: Admin & Settings Testing

Summary

Add Storybook tests for admin and settings interfaces.

Expected Benefits: Improved maintainability and layout consistency.

GitHub Issue: 📚 Admin & Settings Testing

Current Situation

• No Storybook tests and layout inconsistencies.

Objectives

• Conduct a code review, add tests, and fix layout issues.

Subject 11: Migrating to RabbitMQ

Summary

Complete the migration to RabbitMQ for all service communications.

Expected Benefits: Improved resilience, performance, and issue detection.

GitHub Issue: 📚 RabbitMQ Migration

Current Situation

• Some services still use HTTP for communication.

Objectives

• Replace HTTP with RabbitMQ.

Subject 12: Real-Time Data Updates

Summary

Implement real-time data updates for the web application.

Expected Benefits: Enhanced user experience and a modernized platform.

GitHub Issue: 📚 Real-Time Data Updates

Current Situation

• Periodic data pull strategy is used.

Objectives

• Switch to a push strategy and decide on using Knock or an in-house solution for notifications.

Phase 1 - Workload Estimates

Large Topics

• #0 Rethinking File Management
• #4 Unify Conditional Formatting, Mapping, and Formula
• #5 Splitting the Items Table
• #8 Removing Haku

Medium Topics

• #1 Account Creation Tool
• #12 Real-Time Data Updates
• #10 Admin & Settings Testing
• #7 Accessible Product Usage Analytics

Small Topics

• #3 Improved Yuba CLI
• #6 Cleaning Up Feature Flags
• #9 Detail View Refactor
• #11 Migrating to RabbitMQ

Phase 2 - Prioritization Results

• #0 Rethinking File Management was pre-identified as a priority.
• Most voted topics:
  • #2 Resolving Technical Debt
  • #12 Real-Time Data Updates
  • #5 Splitting the Items Table (POC only)
  • #11 Migrating to RabbitMQ
  • #9 Detail View Refactor
  • #6 Cleaning Up Feature Flags
  • #3 Improved Yuba CLI
  • #10 Admin & Settings Testing
• Topics not prioritized for 2025:
  • #1 Account Creation Tool
  • #7 Accessible Product Usage Analytics
  • #8 Removing Haku

Phase 3 - Yearly Timeline