Why Do Data Integration Architectures Fail to Perform?

Industry research reveals that most data integration initiatives fail. Common problems include:

• Late delivery and over budget
• Poor performance of queries and loads
• Unacceptable data quality for accurate analytics and reporting
• False starts, potentially resulting in complete rework

Why does this happen? Can it be prevented? These are the type questions we think deserve your utmost consideration before you begin a large-scale initiative, or attempt to rework an existing solution. Based upon our significant experience working in this market, we believe one of the key issues underlying all these problems is an overall lack of planning, especially in regard to the technical architecture’s planned service delivery levels. With this in mind, Myers-Holum has developed a Performance and Tuning Assessment offering to address this need. We typically provide this service as a courtesy to our customers (i.e. we charge a nominal fee), providing us an opportunity to demonstrate our unique technical expertise and capabilities.

Our Approach to the Problem

• Prepare – During this phase the approach, scope, deliverables and timelines for the assessment are defined and approved.  The team members, both from Myers-Holum and the customer would be identified.  Other pre-projects activities such as scheduling of interviews and organization of a team kick-off meeting help ensure a highly successful engagement.

• Discover – During this phase the team focuses on establishing a solid understanding of the current technical environment and the business drivers and associated data volumes / timetables this environment must support.  This process is based on both a review of existing technical documentation and interviews with technical staff.

• Analyze – During this phase the team will identify the bottlenecks that exist in the various components—technical, system, data and business process—of the environment.  The type of assessment will determine the extent that each bottleneck will be analyzed.  It is also during this phase that the team will begin to evaluate various solution options for addressing the bottlenecks that have been identified.

• Synthesize – During this phase the team will formalize the solution recommendations.  At this point, members of the team will begin preparing recommendations for addressing performance issues.  Once the recommendations have been defined, they will be reviewed with key members of the client’s technical team so as to receive their concurrence.

Client Deliverables

Typical deliverables from this type of engagement can include one or more of the following:

• Findings and Recommendations Covering:
  - Data Architecture
    > Conceptual, logical and physical data models
    > Partitioning, Indexing
    > Real-Time Load Dependencies
  - Technical Architecture
    > Hardware
    > Software
    > Operating systems
    > Storage
    > Network
  - Data Integration Architecture
    > Data movement from source to target
    > Transformations
    > Data Quality
    > Data Cleansing
    > Balance & controls
  - BI Architecture
    > Reporting
    > Decision Support
    > OLAP

• High level remedial activities / projects to resolve performance and tuning bottlenecks

Free Offer to Qualified Clients

We have so much confidence in our broad-based technical abilities that we have a standing offer to qualified larger companies. We will provide highly-skilled resources on a short-term, gratis basis to assist customers having large-scale performance problems. For example, we recently did this at a very large credit institution. At the time they were considering replacing their data integration technology. After we spent two days on-site, we rectified the problems and they are currently well-satisfied with their current technology.