EIQ Product Server Configuration Concepts
EIQ Product Server Configurations
EIQ Product Server Connection Schema Interfaces
EIQ Product Server Configurations in Deployment
This document describes the SmartData Fabric® EIQ Product Server concepts, its various configurations and its deployment options. EIQ Product Server is also referred to as EIQ Server. The two terms are used interchangeably throughout the EIQ Product Suite Help documentation. This document covers the following topics:
EIQ Server components can be configured in multiple ways to provide different functionalities. Understanding the following concepts is important for configuring EIQ Product servers.
Admins use the EIQ Server Configuration Tool to configure and manage EIQ Product servers. EIQ Product servers run as services and they must be running before clients can connect. The EIQ Server Query Tool is a query client tool that can connect to EIQ Product servers for executing queries and returning results.
EIQ Product servers provide standard data access for clients through a Virtual Data Source (VDS) interface. Clients connect to EIQ Product servers similarly to how they connect to standard data source servers through ODBC, JDBC or native connection drivers. They use a VDS like any other standard relational data source.
A Virtual Data Source (VDS) consists of the following:
1) a name for client applications to connect to (similar to data source name)
2) a schema view definition (either a virtual schema view based on a standard data model along with column mappings or the native data source schema),
3) data source(s) registered with EIQ Server
4) depending on the configuration, a pairing of EIQ Indexes with corresponding registered data source
A VDS provides access to a single data source (EIQ SuperAdapter, EIQ TurboAdapter, and EIQ ConventionalAdapter) or multiple data sources (EIQ Federation Server).
An EIQ Product server VDS can provide two types of schema views:
1) a virtual schema view based on a standard data model, or
2) native data source schema
The EIQ Product Server’s SuperSchema mapping feature allows admins to map data source specific columns to standard data model columns for Virtual Data Sources. This hides data source specific schema names. A VDS without a SuperSchema mapping exposes the original data source schema to clients.
Virtual Data Sources hide details of configured data sources behind a SuperSchema mapping. A SuperSchema contains one or more tables composed of column names from a metadata dictionary based on a standard data model that are mapped to data source specific columns containing the corresponding data.
EIQ Product servers execute queries in one of three ways:
1) at EIQ Indexes corresponding to the data source being accessed
2) by passing queries to the data source being accessed
3) by passing queries to other EIQ Product servers
In a typical edge-ware configuration, as an EIQ SuperAdapter or EIQ TurboAdapter (see EIQ Product Server Configurations ), EIQ Product servers execute queries on EIQ Indexes corresponding to data sources and access data sources only for retrieving query results. EIQ Indexes, when cleansed, standardized, and enhanced, enable additional query processing capabilities otherwise not available at data sources.
Admins register data sources with the EIQ Product server to provide the server with necessary information for connecting to data sources, such as, log-on credentials, driver type, and data source schema name. In typical configurations, data sources and their corresponding EIQ Indexes are paired so that EIQ Product server knows which index to use for query execution for a data source. Depending on the configuration, some EIQ Product servers may not have EIQ Indexes. They pass queries to other EIQ Product servers or directly to data sources for execution.
When a client connects to an EIQ SuperAdapter, opens a Virtual Data Source, and submits a query, the EIQ SuperAdapter translates the query from a standard data model to data source-specific schema. It executes the translated query on EIQ Indexes corresponding to the data source. The query on indexes DOES NOT return the data requested by the client. It returns only ROWID values for rows in the data source satisfying query requirements. Then the EIQ SuperAdapter uses the ROWID values to retrieve raw data from the rows in the data source.
EIQ Federation Server passes the client query to other EIQ Product servers (EIQ SuperAdapter, EIQ TurboAdapter, EIQ ConventionalAdapter, and EIQ Federation Server) which each in turn process the query and return the results to the federation server. The federation server combines the results from every EIQ Product server and returns results to clients.
Based on factors, such as the number of data sources being accessed, schema view type, and the component executing the queries, EIQ Server components can be deployed in the following configurations:
Note: EIQ SuperAdapter, EIQ TurboAdapter, and EIQ ConventionalAdapter are collectively referred to as EIQ Edgeware or EIQ Adapters. These three configurations are used in place of conventional adapters in federated data systems. EIQ SuperAdapter and EIQ TurboAdapter use EIQ Indexes for query execution. EIQ ConventionalAdapter is similar to conventional adapters by other vendors and passes queries directly to the data source for execution.
REGDB Data Source: Additionally, all of the above EIQ Product server configurations allow their clients to connect to data sources registered with them directly in a special pass-through manner using an "REGDB" data source connection. An REGDB data source connection provides clients access to a single data source with no EIQ Indexes and no standard data model-based virtual schema view. Clients can view native data source schema and execute queries at the data source directly. This can be useful for updating data sources.
The following table summarizes the major features of each EIQ Server configuration and REGDB connection mode.
Table 2: EIQ Product Server configurations and connection options for EIQ Server components
|
Feature |
EIQ Federation Server™ |
EIQ SuperAdapter™ |
EIQ TurboAdapter™ with optional data clean-up and transforms |
EIQ TurboAdapter™ |
EIQ ConventionalAdapter™ |
REGDB Connection |
|
Typical Use |
Accessing multiple data sources through EIQ Adapters |
Accessing a single data source |
Accessing a single data source |
Accessing a single data source |
Accessing a single data source |
Accessing a single data source |
|
Has EIQ Indexes for Query Processing? |
No |
Yes |
Yes |
Yes |
No – queries are passed through and executed at the data source |
No – queries are executed at the data source |
|
Schema |
Standard Data Model-based SuperScehma or data source Native schema |
Standard Data Model-based SuperSchema |
Data source Native schema |
Data source Native schema |
Standard Data Model-based SuperSchema or data source Native schema |
Data source Native schema |
|
Standardization/Cleansing Data Transforms at EIQ Indexes? |
N/A |
Yes – as per standard data model |
Yes – as per requirements |
No |
No |
No |
|
Applies Data Transforms to Results |
Not currently (likely in future) |
Yes |
Yes |
No |
Yes |
No |
|
Supports Link Indexes |
Yes |
Yes |
No |
No |
No |
No |
|
Query Support |
Federation SQL Syntax |
SuperSchema level SQL Syntax |
WhamTech ANSII 92-99 SQL Syntax |
WhamTech ANSII 92-99 SQL Syntax |
Data source SQL Dialect |
Data source SQL Dialect |
|
Client Connection Schema Interface |
StandardDataModel or Native |
StandardDataModel |
Native |
Native |
Native |
Native |
EIQ Product Server configurations can be flexible. Apart from the ones mentioned above, users can have special case configurations within a single instance of an EIQ Product Server:
1) A special configuration can be a hybrid of EIQ SuperAdapter and EIQ Federation Server configurations - EIQ Federation Server with local 'Data Source - EIQ Index' pairs for one or more Virtual Data Sources (VDSs).
2) EIQ SuperAdapter instances are generally dedicated to a single data source. However, a special EIQ SuperAdapter configuration may have more than one 'Data Source - EIQ Index' pair in a single Virtual Data Source (VDS) for accessing data from multiple disparate data sources. In this case, all the EIQ Indexes reside locally with the server where queries are executed. Queries formed based on virtual schema submitted to EIQ Indexes after translation to data source specific schemas.
3) A special case EIQ SuperAdapter or EIQ TurboAdapter may 'attach' EIQ Indexes belonging to multiple data sources and execute complex JOIN queries across the data sources as if all the tables belong to a single data source. In a VDS with 2 or more 'Data Source - EIQ Index' pairs, the EIQ Indexes can be 'attached' so queries can be formed using any of the tables from the data sources. As usual, queries are executed at the EIQ Indexes and results retrieved from respective data sources.
For an EIQ SuperAdapter, the queries can be formed on virtual schema views based on a standard data model. To enable this, the server needs additional configuration for column mapping and PK-FK relationships on tables across data sources. As expected, EIQ TurboAdapter clients can continue to use the original data source schemas and have the flexibility to form queries as they wish.
An example scenario where 'Attach' mode can be useful is described below:
Data Source 1 residing on DSServer 1: Employee Data
· Employee Table
· Salary Table
Data Source 2 residing on DSServer 2: Sales Data
· Customers Table
· Sales Table
A single instance of an EIQ SuperAdapter or EIQ TurboAdapter can be configured to have these two data sources under a single VDS by registering both 'Data Source - EIQ Index' pairs. Clients can connect to the VDS and execute queries on tables across data sources. An example query enabled by this configuration could be: 'Find the total sales grouped by the sales person's salary grade.'
Note: The clients must set the connection mode to 'Attach' mode if they are to exploit the 'Attach' configuration. See below for more details regarding connection modes.
EIQ Federation Server provides external JOIN functionality across data sources through separate EIQ SuperAdapters (one dedicated per data source) which may require necessary data to be transferred across EIQ SuperAdapters. In comparison, the 'Attach' feature provides the following benefits:
1) flexible queries because queries won't be limited to a flat virtual schema view like with an EIQ Federation Server
2) faster query execution as queries are executed locally on a single machine on the attached EIQ Indexes.
However, depending on the data source locations and owner preferences for data protection, it may not be possible to do 'Attach' in all cases. Also, 'Attach' may not be as scalable as the EIQ Federation Server option for all EIQ Indexes must be local to a single instance of an EIQ Product server.
By specifying a connection schema interface, clients can often override the default EIQ Product server configuration schema interface for a connection session. The available connection schema interfaces are:
1) StandardDataModel - A client connecting to an EIQ SuperAdapter VDS must use this schema interface to get the standard data model-based schema interface and submit queries on this schema.
2) Native - Native is the default schema interface option for any client connecting to an EIQ TurboAdapter VDS. Clients connecting to an EIQ SuperAdapter VDS using the Native schema interface option can bypass the standard data model-based schema and submit queries on the data source native schema as if the VDS is configured as an EIQ TurboAdapter. Other clients will continue to see the VDS as an EIQ SuperAdapter.
3) NativeWithAttach - If an EIQ adapter is configured with Attached indexes (multiple EIQ Indexes attached as if they are one), then clients must use this connection option to see all the attached tables in their native schema.
4) StandardDataModelWithAttach - If an EIQ Adapter is configured with Attached indexes, then clients must use this connection option to see all the attached tables in terms of standard data model-based schema.
Below are some example deployments of various EIQ Product servers in relation to data sources, applications, and third party federation servers. Note that EIQ Federation Servers present a single view of multiple data sources and EIQ Edgeware servers (EIQ SuperAdapter, EIQ ConventionalAdapter, and EIQ TurboAdapter) provide access to a single data source each.
Figure 5: An example EIQ Product deployment in a shared-nothing configuration.
Figure 6: An example EIQ Product deployment along with third party adapters.
Figure 7: An example EIQ TurboAdapter deployment
Figure 8: An example of flexible EIQ Product Server deployment in 'Attach' mode.
Copyright © 2019 , WhamTech, Inc. All rights reserved. This
document is provided for information purposes only and the contents hereof are
subject to change without notice. Names may be
trademarks of their respective owners.