THE CHALLENGE
Our customer had been using the Business Intelligence Platform on Teradata to host sensitive data that was regularly accessed and updated by third-party vendors. But the platform’s limited monitoring capabilities were difficult to manage and did not meet the bank’s growing needs. Meanwhile, the solution had become cost prohibitive. To save money and meet their business requirements, the customer needed to migrate from the legacy platform to a modern unified data platform.
THE SOLUTION
In addition to migrating more than 1,000 highly sensitive datasets, EveryIT migrated 23 regulatory reporting functions onto the modern unified data platform. To expedite project delivery and validate data accuracy, EveryIT built an automated tool that compared the datasets moved from the legacy platform to the target platform. The EveryIT team chose a third-party tool to convert Teradata SQL to Sparkola SQL, further expediting the project. To enable secure user access on the new platform, EveryIT designed a custom access control system based on a proprietary tool our customer had built on top of Spark.
Due to the sensitive, business-critical nature of the data and systems as well as regulatory guidelines, EveryIT followed strict protocols for the deployment and developed plans to manage capacity, data involvement, and other conflicts. Additionally, the team developed systematic plans for migrating data that was heavily dependent upon other systems while contending with a continuously evolving system during the migration. Finally, EveryIT sanitized the customer’s environment, resolving 120 critical issues.
Modern unified data platform architecture
THE RESULTS
The migration to a modern unified data platform provided multiple benefits for our customer:
- Reduced costs by 80%.
- Boosted efficiency of their entire system by 40%.
- Increased control over management of third-party rights.
- Gained the flexibility to add changes as needed.
- Expanded data visibility and monitoring capabilities.
“Fantastic effort and teamwork to get this done. It has been a challenging task, especially in this COVID period, but your diligence and collaboration have pulled it off. Really appreciated.”
– Banking customer
TECHNOLOGIES USED
Sparkola, Superset, Alluxio, Apache Airflow, Grafana, AWS S3 bucket as a data store, and Presto, Deep Monkey, and Compiler Works.