iWay Big Data Integrator provides a simple interface for Hadoop and Spark data ingestion, transformation, changed data capture (CDC), enrichment, and cleansing of big data sources. It can ingest data from structured relational data sources (RDBMS), defined Hadoop, and other NoSQL sources and file formats (Hive, Avro, sequence files, Snappy-compressed CSV, etc.). It can also capture and handle event or streaming events sent via Apache Kafka, Flume, and Spark streaming to enrich, or combine or cleanse event data.
Additionally, data mastering may be more suitable than ETL tools for situations that require access to complete, consistent data in real time. Omni-Gen Master Data Management Edition provides seamless integration and the ability to interact with existing systems and data structures to enable the rapid consolidation of millions of records according to easily defined business rules. Unified and validated master data is instantly available to a wide range of enterprise applications, such as enterprise resource planning (ERP), customer relationship management (CRM), self-service portals, analytical tools, data warehouses, and other internal systems.