We recently started using Splunk in a customer project in addition to an ETL route with Informatica PowerCenter - here I would like to talk in particular about the possibilities that this combination offers.
The tools
But first a brief introduction to both tools. Informatica PowerCenter is a classic ETL tool (extraction, transformation, loading) and has been on the market for many years. It has a repository-based approach and stores the information about the developed software artifacts, relationships and executions in a relational database. You can find more details about PowerCenter in our previous blog posts.
Splunk Enterprise is the product of the Splunk company of the same name. The product can process and visualize pretty much all data provided by machines (log files). Almost any source can be connected. Classic relational databases can also be integrated. And this is where the Informatica PowerCenter Repository comes into play. This is because it is installed and operated on a relational database (Oracle, IBM DB2, Microsoft SQL Server or Sybase ASE).
Possible areas of application
The data stored in relational databases can be queried using SQL (Structured Query Language). Interesting database queries can be created and executed against the Informatica repository. For example, different environments can be queried in this way for adjustments to the software versions and reports can be generated. If, for example, the deployment takes place on a test environment, Splunk can check the time stamp at regular intervals and generate a deployment report - this can also be sent by e-mail or displayed in a dashboard - Splunk provides these functionalities directly.
The log files of individual PowerCenter workflows or sessions from different environments can also be integrated into Splunk using agents in order to evaluate them. This allows you to search for specific error patterns across all log files collected in a specific time period. Direct access to the environment is also not necessarily required if a workflow termination needs to be analyzed.
However, it is important to know that everything that ends up in Splunk can be analyzed. If your Informatica log files contain personal data, for example, these must be anonymized and/or access to the search must be restricted.
In summary, it can be said that the combination of Splunk and Informatica harmonizes very well with each other and offers Splunk developers many possibilities to make their daily work easier and to get important information to the right recipients.
