Inquidia Reflex: Standards and Best Practices for Pentaho Data Integration

Wednesday, April 6, 2016 - 11:15

The Pentaho Data Integration (PDI) tool has been around for ten years and offers a wide range of abilities to blend and relate disparate data sources.  These sources can be modeled into a centralized point of access for a company’s users to not only answer every day operational questions, but to also highlight trends, and build analytics that can assist in overall growth.  The open source tool offers a user interface that is straightforward and simple to use; however that doesn’t mean it should be without good planning and coding rules when it comes to enterprise solutions.

For instance, there are many different ways to connect to your data; JDBC, JNDI, or ODBC.  Which is the right one for your environment?  Should variable usage be employed via kettle.properties or some other mechanism?  Do we make these connections shared across our developer community?  What’s the naming convention?  

For any toolset that offers so many options, there ultimately are a set of standards and best practices that can be calculated based upon experiences.  Inquidia Consulting has also existed for over a decade, and has been using the PDI tool all along on numerous projects across a broad spectrum of industries.  Over the years we have seen the capabilities grow greatly within, and have contributed to that growth through numerous JIRA’s and plug-ins submitted all based upon experiences in the field.  

With such a vast knowledge base, we've reviewed these projects to gain an understanding of what patterns have worked best for our consultants while providing the most value to our client’s needs and challenges.  This list points to many paths of standards to ensure that the best possible solution is utilized and can be applied to many different types of environments.  

So with all that in mind, our Inquidia Labs development team collaborated on the development of documented standards and best practices for our entire consulting organization. These standards fall into our broader methodology and approach and are part of our internal Inquidia University curriculum.

We call these development best practices Inquidia ReflexSM. These standards provide direction on approaches to handle the use of the Pentaho technology to the fullest, including approaches for:

  • Data Connections

  • Logging

  • Joins

  • Error handling

  • Lookups

  • Sub-Routines

And beyond these topics we include practices to organized data processing on subject areas sometimes overlooked, such as:  

  • Job & Transform Naming Conventions

  • Solution Repository Folder Organization

  • Job Entry & Transform Step Naming Conventions

By instilling these practices, we’ve been able to achieve a more uniform and streamlined development environment that is more easily transitioned & supported from one developer to the next.  Auditing and troubleshooting capabilities are also enhanced based upon a known naming and coding set of rules.

Want to know the difference between variables and parameters?  How to use each?  And when?  Inquidia Consulting has the answers in hand with our “been there done that” Inquidia ReflexSM Standards and Best Practices for Pentaho Data Integration.  When organizations work with our Pentaho experts,  we come equipped with a wealth of knowledge on the most sustainable Pentaho development methods. We’ve established these as a tried, true, and documented methodology to ensure a consistent, efficient, and effective deployment that will benefit & support your company’s current and future requirements.

If you’d like more info on Inquidia ReflexSM and how we make it work for you, drop us note at info@inquidia.com. We’d be glad to share with you how we approach successful Pentaho development.

 

Contact us today to find out how Inquidia can show you how to collect, integrate and enrich your data. We do data. You can, too.

Would you like to know more?

Sign up for our fascinating (albeit infrequent) emails. Get the latest news, tips, tricks and other cool info from Inquidia.