Agents

NEOS BI Framework contains several types of agents, which can be independently tuned and monitored. They provide administrators with powerful tools for system health maintenance.

They also enable dynamic execution of processes which is one of topmost needed features in large DW systems, with complicated interdependences – system checks those dependences and execute processes as soon as all prerequisites are met (with respect to balancing system load in defined limits). This feature, which avoids hard-coded ETP execution plans, brings significant performance boost, and provides optimal load on system resources, what is not possible to achieve manually.

Other types of agents provide logging capabilities, and automated cleaning possibilities on lowest granular levels. They enable maintenance of ETL metadata and DW data on lowest granular level, with definable retention periods for object or object group. For example, it provides possibility to define time window for cleaning of process metadata, automatic intelligent partition deletion in Data Staging Area with respect to retention period (day, week, 10 days, month, etc.) and according to data processing state (successful further transformation).

Agents features include:

Plan & Execute

  • RWYC Principle Based Execution
  • Dynamic Execution Plans
  • Based on Prerequisites Automatically Identified from ETL Repository
  • Configurable (Priorities, Execution Times.. )

Clean & Utils

  • Automatic Cleaning of Processed Data (based on ETL execution status)
  • Automatic and ETL aware Statistics Calculation
  • Configurable on Lowest Level

Logs & Stats

  • Additional Process Execution Statistics
  • Detail logs of execution times, type of operations
  • Base for Performance Issues Analysis