SimplifyVMS - 173tech

Data Roadmap

Turning reporting and analytics as a core SaaS feature

Objective

Build complex multi-tenancy infrastructure for client reporting.

Obstacle

An expensive data stack, non-standard requirements and bespoke reporting.

Outcome

Over $300k saved on annual data stack costs and an in-house team embedded.

Background

SimplifyVMS is a provider of vendor management systems for temporary staffing needs with clients like EasyJet, Mariott and PWC. Its platform offers extreme flexibility to clients, enabling efficient management of recruitment services for contingent workers.

The team wanted to enhance the reporting function offered to clients by providing them with 100% customisation of their own dashboards and the ability to drag-and-drop bespoke reports. This meant setting up multi-tenancy infrastructure and bespoke modelling pipeline for each individual client – without costs ballooning. They turned to 173tech with this big technical challenge…

Challenges

Costly Data Infrastructure: SimplifyVMS’s existing reporting infrastructure relies on Databricks and EC2, costing approximately $30,000 per month. The complexity of the architecture results in inefficiencies, making it difficult to scale. Moreover, the infrastructure requires substantial maintenance, with frequent memory issues arising in Kafka and Databricks workers.

High Manual Intervention: Client onboarding takes months due to the need for manual code creation. Each client has a separate Git branch, requiring individual updates for new metrics and features. Additionally, the lack of automation in data reporting leads to increased manual work and redundancy in code, delaying insights for clients.

Solution

Redesigned Data Architecture: To address these challenges, SimplifyVMS explored multiple data architecture options, including Redshift, Databricks, Snowflake and Clickhouse. The selected solution focuses on implementing federated schemas in Redshift to enable real-time data access, introducing dbt for automated data transformation to reduce manual intervention, leveraging Airflow as an orchestrator to streamline ETL processes and developing a multi-tenant schema strategy to support custom client requirements efficiently.

Enhanced Data Processing and Governance: The new solution introduces a structured data modelling layer to simplify reporting and minimise the need for manual SQL joins. Additionally, governance policies ensure secure access controls and data segregation per client, improving compliance and security.

Impact

Infrastructure Setup: Redshift was implemented as the central reporting warehouse, with dbt integrated to automate data transformations. A federated schema system was designed to enable seamless data access across sources, while automated onboarding workflows reduced client setup time from months to just a few days. To enhance performance, materialised views were introduced to optimise query speed and minimise reporting lag. As a result, data refresh rates improved from hourly to near real-time, empowering clients with faster, more informed decision-making.

Knowledge Transfer: We worked closely with SimplifyVMS’s internal team to enhance their understanding of data architecture, analytics, and best practices. By providing guidance on common data challenges, we empowered them to troubleshoot issues effectively, optimise their data workflows, and implement scalable solutions.”

We collaborated with SimplifyVMS to enhance their internal team’s expertise in data architecture and analytics. Through targeted training and hands-on support, we helped them resolve data integration challenges, optimise reporting workflows, and implement scalable solutions tailored to their needs.

Future-Proof: Through a strategic overhaul of its data architecture, SimplifyVMS achieved a scalable, cost-effective and automated reporting system. The new infrastructure enables the company to onboard clients faster, reduce costs and provide faster insights.