EIQ Adapters™ and sub-middleware EIQ Federation Servers™ form the core of SmartData Fabric® EIQ Products™, and include the tools and processes to:
- Discover data sources
- Profile data
- Classify and secure data
- Develop data quality transforms
- Map indexed and source data to virtual Business Data Views (BDVs)
- Build and maintain indexes, and
- Process queries
EIQ Adapters™, in particular, EIQ Indexed Adapters™ and EIQ Hybrid Adpaters™, offer significant advantages over conventional federated adapters and these advantages are discussed in the following documents:
- EIQ Indexed Adapters™ vs. Conventional Adapters in Federated Data Systems (one-page summary)
- EIQ Indexed Adapters™ vs. Conventional Adapters in Federated Data Systems (white paper)
All EIQ Products are independently configurable and do not have to adhere to any fixed hierarchy or network structure. The only configuration that is defined in an EIQ Adapter is the data source and for an EIQ Indexed Adapter or EIQ Hybrid Adapter, the indexes, that an EIQ Adapter connects to. The same is true for a EIQ Federation Server where the EIQ Adapters and other EIQ Federation Servers it connects to, are configured. This allows for great flexibility to add or remove EIQ Products, either vertically or horizontally, with minimal impact, plus, allows high performance, scalability, load balancing, failover and redundancy options. This flexibility also introduces LIFO and FIFO query execution considerations, as the same query can be submitted multiple times from different configurations to the same EIQ Product, but these can be taken care of through simple rules. FIFO tends to be faster and more manageable.
The three EIQ Adapter and one EIQ Federation Server products currently offered are:
- EIQ Indexed Adapter for data source access – with indexes and optional BDV mapping
A superior alternative to conventional federated adapters, offered on a data source basis, includes the tools mentioned above and query transforms for a specific data source and schema, which is generally mirrored in the indexes. Results from an EIQ Indexed Adapter are provided in the same SDV format as results from other EIQ Adapters, each accessing their own respective individual data sources. Data that is not indexed and does not need to be queried can also be read and transformed as part of results data. EIQ Federation Server sub-middleware and/or other third-party products can combine results from multiple data sources and associated EIQ Adapters, other data sources and associated third-party adapters, and other EIQ Federation Servers. - EIQ Conventional Adapter™ for data source access – conventional, without indexes, but with optional BDV mapping
In a situation where a data source has query load capacity and there is no need or option to cleanse or transform data for queries, an EIQ Conventional Adapter can be used and is similar to a third-party conventional federated adapter. Queries are submitted to a BDV, and transformed by the EIQ Conventional Adapter, however, queries are submitted to, and executed by, the data source – not external indexes. Results data can be cleansed, transformed and standardized. - EIQ Hybrid Adapter for data source access – combination of EIQ Indexed Adapter(s) for data that needs indexes, and EIQ Conventional Adapter(s) for data and data sources that do not need indexes but need to be and can be queried, with optional BDV mapping for all data
Most data in many data sources, e.g., transactional systems, does not need the power of indexes for data issues, as it is generally non-human-generated. However, other data, e.g., personal, product and organization data, that tends to be slow changing over time and human-entered, needs pre-processing to address issues, e.g., quality (missing, typos, transposed, abbreviated, nickname, etc.), data type and standards. This type of data is needed for clean, accurate, complete and secure queries and results, and Master Data management (MDM), which needs to track changes in this type of data over time, ideally, in near real-time. - EIQ Federation Server for all EIQ Products integration – no direct data source access – with BDV mapping
Virtual data integration, sharing and interoperability query management sub-middleware to access multiple EIQ Adapters and other EIQ Federation Servers. EIQ Federation Servers do not have their own indexes, but may perform some query processing, e.g., joins across multiple adapters to data sources, or results transformation, but can distribute queries and combine results data from other EIQ Products, including other EIQ Federation Servers. As EIQ Federation Servers can access other EIQ Federation Servers as though they are EIQ Adapters, and can be independently configurable, they allow for high performance, scalability, load balancing, failover and redundancy, and multiple levels of federation, e.g., local, department, branch, company, state, region and country.
The following table lists the major feature differences among the products:
A detailed description of EIQ Adapters is also available at EIQ Adapters in detail.
More information on WhamTech products, click here.