RCG|enable® Data Ingestion

Highly Scalable, Distributed, Secure and Fault Tolerant

RCG|enable® Data Ingestion is a fully integrated, highly scalable, distributed and secure solution for managing, preparing and delivering data from a vast array of sources including social media, mobile devices, smart devices and enterprise systems. The framework eliminates the need for IT professionals to become experts in Hadoop eco-system technologies and languages and speeds time to delivery at reduced costs by simplifying and standardizing data management and data workflows.

The framework supports data sources (structured, semi-structured and unstructured) and targets in traditional enterprise systems, external systems, and the Hadoop platform eco-system.

RCG|enable® Data Features

  • A single consistent method for capturing data
  • The ability to quickly add new data sources and targets
  • A foundation of open source technologies
  • A Highly scalable, distributed, secure and fault-tolerant architecture
  • A component-based architecture that enables plug & play of new connectors, transformers, etc.

RCG’s Web Portal provides graphical drag and drop capabilities allowing developers to specify workflow and data transformations. Support is provided for Generic Formats (Database results, Delimited, JSON, XLS, XML Thrift, Avro, PDF), Hadoop Formats (Parquet, RCFile, Sequence File) and Industry Specific Formats (Accord, LAS, FIX)

Execution Engine

The RCG|enable® Data Ingestion engine captures, transforms and moves data. It can be initiated through a command line, Web Portal or can be scheduled to run automatically. Features include:

  • Parallel and distributed
  • Visual Drag & Drop interface
  • Portable, lightweight and secure
  • Support for batch, micro-batch, near real-time and real-time delivery modes  
  • Metadata and lineage aware
  • Secure end-to-end data routing encryption and compression
  • 100% Open Source
  • Over 100 supported endpoints
  • Flexible Transformations
  • Aggregations – Filters and Mappings
  • Plugin architecture
  • Distribution Agnostic
  • Supports for structured, semi-structured and unstructured data

Source and Target Data and Systems

RCG|enable® Data has multiple integration points with the leading Hadoop platforms using key open standards. Source and target data and systems are supported through connectors. Connectors are available for many types of data and systems and new ones can be added to the framework quickly and economically. Supported connectors include:  

  • Hadoop – HDFS, Hive, Hbase
  • Database – Teradata, Neteeza, SQLServer, MySQL, Oracle, DB2
  • NoSQL – DataStax, Cassandra, MongoDB, Couchbase
  • Social Media – GNIP, Datasift
  • Search – Solr, ElasticSearch
  • Messaging – Tibco, MQ, Kafka, Active MQ, RabbitMQ
  • File Systems – Generic OS, FTP/SFTP, Splunk
  • Streaming – Spark Streaming, Sockets
  • Integration with Camel to support hundreds of different endpoints

Subscribe to get the Latest Updates

Enter your email address below to get the latest news and updates from RCG.