SEQUEL Data Warehouse Picks Up Where RODIN Left Off

Not every IBM i shop needs a big data warehouse to store and analyze information about their operations. But for those organizations whose data needs go beyond basic reporting capabilities, it is good to know there are tools like HelpSystems‘ SEQUEL Data Warehouse, which was formally called RODIN and which the company just released at version 8.0 with substantial new functionality.

It has been a year-and-a-half since HelpSystems acquired Coglin Mill, the Rochester, Minnesota, based company that developed RODIN. Once dubbed “the IBM i platforms best-kept secret,” RODIN provided enterprise-grade data preparation capabilities and extract, transform, and load (ETL) functionality that hundreds of IBM i shops have relied upon.

Now called SEQUEL Data Warehouse (SDW), the software has been fleshed out at version 8.0 with significant new functionality. Enhancements in the areas of change management and error reporting have already garnered interest from the RODIN installed base, says Alan Jordan, senior development manager.

While the SDW software is stable and mostly bug-free, errors can crop up during data warehousing activities, according to Jordan. This is especially true when you consider that data warehouses are typically used to consolidate operational data from multiple systems. No matter what mechanism is used to move that data–whether it’s ETL, data push, continuous change data capture (CDC), or all of the above–the process is dependent upon other servers, database management systems, and network links to operate correctly.

“While our customers often tell us RODIN/SDW is one of their most stable and reliable products, things can go wrong,” Jordan says. “Remote connections may fail due to network issues, security issues, or application changes. During ETL, business rules may reject bad data. All of these–and more–need to be monitored and we need to make it easy for the responsible parties to get an email notification, with as much info as possible for them to take action.”

RODIN/SDW already had some error reporting, Jordan says. But as more functionality was added to the software, it became apparent that the software needed a better framework to make it easier to extend event notifications to data warehousing project manners in a consistent manner, he says.

“The re-design has also allowed us to add notifications in many new areas,” he says. “For example, if, while developing the data warehouse tables and ETL processes, a build fails, the relevant listings can now be automatically sent via email to the team lead for investigation.”

Similarly if a problem should crop up in SDW itself–a rare but theoretically possible occurrence–the joblog will automatically get sent to someone to investigate, potentially even to the HelpSystems support team. “Imagine getting a call from your vendor with the solution to the software problem you may not yet be aware of!” Jordan says.

The event notification capabilities arrive via the embedding of another HelpSystems product, called ESEND, into SDW. ESEND allows individual users to subscribe to specific events or to subscribe to events of a specified variety. It also automatically includes attachments, such as the error report or joblog, to the emails.

The second main area of enhancement in SDW 8.0 focuses on improving the management of change within an active data warehouse project. In a large data warehouse environment, Jordan explains, it’s not uncommon to see multiple developers working on different pieces of the puzzle. It’s not uncommon to see 500 data warehouse and data mart tables, with indexes, materialized query tables (MQTs), and views built over them, all with their own ETL feeds and metadata logs.

“There can easily be thousands of moving parts to manage in the development process,” Jordan tells IT Jungle. “Then consider the ongoing maintenance, with changes, enhancements, additions etc. In a traditional development environment (i.e., writing RPG code), shops use Aldon, Turnover, or other change management packages to manage all of this. But that is not practical with SDW.”

So Jordan and his team decided to implement better change management in the software. The rebuilt change management module starts with the concept of a “project,” which must be defined before any development work can begin. All activity against the project is tracked, and each developer is assigned a default project. Even if developers don’t sign into a project, the software tracks the changes. And all attributes of the data element, such as type and length, are now monitored.

“Our previous implementation of this was enough for most of our customer base,” Jordan says, “but we’ve recognized we could do better, and we have, by adding full ‘project’ support. Now a project manage has better tools to see what objects are associated with a small one-day project, or a longer term huge effort. Every development process/step is logged against a project, and it is easy to see all associated definitions, or promote them as a group.”

A number of other enhancements made it into this release, including improvements to the graphical user interface (GUI), which the product has offered for the past decade. Jordan expects many more enhancements to SDW in the coming years.

“We have always had a lengthy roadmap of features we want to add to the product, and that investment will continue under HelpSystems,” he says. “I would say the product roadmap will likely never be complete, as there will always be new directions and concepts that customers will request.”

RELATED STORIES

HelpSystems Grows With RJS And Coglin Mill Acquisitions

Coglin Mill’s ETL Makes Data Warehousing on IBM i More Attractive

Coglin Mill Keeps Pace With BI, Data Warehousing Demands

Coglin Mill Debuts Lower Cost Versions of ETL Tools

Data Warehouses: Know One When You See One?

IBM Updates DB2 Web Query Business Intelligence Tool