Bare metal migration to cloud data warehouse

Performance was critical to the business. With increased workloads, hardware and escalating license costs, they engaged OmniArcs to migrate their high‑performance database to Amazon Redshift. Our experience at Full 360 laid the foundation.

The Problem

The firm was running an on‑premises database solution for ad placement. As workloads increased, hardware and license costs escalated. They chose to migrate their high‑performance database to Amazon Redshift.

The New Architecture

The data warehouse running on Amazon Redshift is part of a larger solution running on AWS:

• The process begins with more than a billion raw log entries per day, which are ingested in near-real-time using Amazon Kinesis.
• Event data is stored in Amazon S3, which contains petabytes of such data at any given time. A single 30-day period may examine 10 terabytes.
• Nightly Apache Spark jobs, which are scheduled using Apache Airflow and run on Amazon EMR and Amazon EC2, handle the ETL process and load the data into Amazon Redshift.
• The data warehouse also consumes reference information that is stored in Amazon Relational Database Service (RDS) and synchronized to Amazon Redshift using FlyData, an Amazon Redshift Partner.

RESULTS

With OmniArcs (informed by our Full 360 heritage), the team migrated the data warehouse workload from Vertica to Amazon Redshift quickly and smoothly, without diverting internal engineering resources. The new analytics platform is more scalable and less expensive, with improved availability and reduced day‑to‑day administration.

Key benefits include:

• On‑demand scalability.
  The team no longer worries about analytics platform capacity and can increase nodes via console or API as needed.
• Lower costs.
  Moving aggregation data to Amazon Redshift avoided a large capital expense and reduced operating costs significantly.
• Less system administration.
  As a fully managed service, Amazon Redshift reduces operational overhead, freeing engineers to deliver new business value.
• Improved availability.
  Managed services decreased downtime and interruptions to business decision‑making.

While the above benefits are powerful on their own, the advantages of working with OmniArcs were also noteworthy: a guaranteed deliverable while internal teams stayed focused on business objectives.