Malware Threats and Cyber-Recovery on IBM i

May 26, 2021 Brian Barth

Cyber threats have evolved from the traditional vectors of theft and direct attacks to more financial driven cyber destruction and extortion, especially via ransomware. Hospitals, manufacturers, universities and financial institutions have all been attacked in the last year, and the rate of attacks is increasing as networks become more accessible to support remote workers. The Colonial Pipeline attack has demonstrated that even the critical infrastructure that supports our supply chains is not invulnerable.

In 2021, more than three quarters of attacks are financially motivated, and over half of those attacks are ransomware – an attack in which files are typically encrypted in place and renamed, making them inaccessible until a payment is made for a decryption key.

However, payment does not guarantee access to data will be restored. Some estimates are that over half of companies paying ransom for data access are unable to recover the impacted system. Data encryption provides no defense against a destructive attack. Traditional disaster recovery and high availability technologies will not prevent a widespread attack and will typically replicate the damage to the disaster recovery (DR) system or high availability (HA) cluster.

Cyber Insurance may provide some compensation for the costs of recovery and will require protocols for security – a well thought out security policy for the IT resources, including network and hosts, is critical. It must be documented and communicated to all users, but it cannot succeed every time – insider threats, human error, and system complexity still present challenges that can overwhelm corporate resource.

IBM i has traditionally been regarded as a secure platform, both in terms of the inherent security capabilities of an object based operating system, and the obscurity of running EBCDIC encoded programs and datasets. Malware is usually transmitted in ASCII stream files, which cannot be executed directly on the IBM i. However, the Integrated File System (IFS) offers a stream file repository that can host and hide malware, and corruption against the operating system can spread throughout the IFS, especially if users are allowed to mount IFS shares, or upload files directly. Additionally, stream files are used for many ancillary programs on the IBM i, such as Java, Web servers, and other commonly used utility applications.

Ensuring that the IFS is properly secured is the first step toward ensuring that business is not crippled by a malware attack on the IBM i. Misconfigured directory permissions or open network ports can allow unintended access to the files and directories that inherit the permissions, including QSYS.LIB – the core of the operating system. Additionally, misapplied permissions may permit destructive access to socially engineered or disgruntled employees. Regular security reviews and security auditing can serve to help protect against this type of attack vector.

Anti-malware products can also assist in early detection and protection from malicious activities. While IBM does not provide this type of scanning directly, the IBM i/OS does provide operating system hooks that enable third party virus scanning products to detect and prevent attacks: See this link from IBM for more on that.

After a successful attack, the choices are limited to paying and praying, or recovering data from the most recent archive. For this reason, it is critical that backup analysis and review are conducted on a regular basis. IBM Backup and Recovery Media Services (BRMS), the most widely used enterprise backup solution for the IBM i, provides a recovery report and other analysis tools that are critical in ensuring that a “bare metal recovery” can be accomplished in the event of a successful attack.

Over the last five to 10 years, many IBM i environments have moved from Physical Tape Libraries (PTL) to Virtual Tape Libraries (VTL). This approach has many inherent advantages over physical tape libraries. Tape cartridges and the drives and libraries necessary to support them are mechanical devices, and prone to single points of failure, usually when it is critical. Both the libraries and the tapes themselves are bulky and require off site storage, which creates security and access challenges, in addition to the expense of secure storage; while virtual tape can be deduplicated and replicated to other secure facilities. Additionally, the number of virtual devices and backup streams can be easily augmented by simple configuration changes on the VTL, for performance tuning or supporting additional environments. Backup appliances may also support concurrent backup of Open Systems network streams such as CIFS and NFS, in addition to providing VTL capabilities.

Both physical and virtual tape backups are subject to the same type of attack vectors that can destroy production data. Human error, social engineering and disgruntled employees leave either technology susceptible to the destruction of the data that is needed to recover from an attack. A secure copy in an inaccessible location is critical.

While creating a secure copy of all IBM i backups for cyber recovery is difficult in a physical tape environment, requiring additional handling and another LPAR or system to create the copies, virtual tape can vastly simplify the process.

Cyber Recovery for IBM i VTL is a solution developed to automate the process of creating an additional secure copy of backup data from virtual tape library cartridges. Currently offering support for VTL on DellEMC PowerProtect, Cyber Recovery for IBM i VTL automatically manages the creation of retention locked, immutable copies on an “air gapped” platform, and can be combined with the PowerProtect Cyber Recovery solution for the protection of other data streams.

To provide secure replication, off of the production network, a direct connection from the production VTL to a Cyber Recovery “vault” is configured on a separate, non-routed network, firewalled at the target against all but replication traffic. Additionally, this replication connection is disabled by the management component of the vault, except during the brief period that replication is actually in progress. Once replicated to the vault, a retention locked copy of the data is created, according to a policy managed by the CR for VTL Management Server, which also manages the expiration of retention locked copies. Typically, retention locked copies are retained from two to four weeks, to ensure that previous backups are available if the current backups are also compromised.

Recovery may be accomplished by replicating the data out of the vault and back to production, where it is restored, or optionally, the environment may be restored to a recovery system in the vault. Often, the Cyber Recovery vault will be located at a DR center, as the production network may be inaccessible while forensic investigations are carried out at the production site. Data diodes – one way fiber optic switches – can be implemented to allow alerts from the vault to be forwarded to the production network, while preventing any access to the vault network.

The National Institute of Standard and Technology (NIST) has developed a Cyber Security Framework consisting of five components: Identify, Protect, Detect, Respond and Recover. The implementation process for a Cyber Recovery Vault reflects this framework:

• Identify: Key assets are identified for backup, determined by criticality, risk, and governance requirements.
• Protect: Security analysis is conducted to define recommendations for measures to secure and control access to key data.
• Detect: Recommendations are made to implement tools that will help detect and recognize events as they occur.
• Respond: Identify the protocols and procedures needed to contain an limit cybersecurity events.
• Recovery: Implement and document the Cyber Recovery Vault and test recovery procedures.

After the analysis of the backup process and security configuration, critical systems which must be recovered for business continuance are identified and backups are configured for replication into the vault. The vault components, usually in a dedicated and physically secured rack, consisting of backup appliance, switch, management host and firewall, are configured with automated policies to manage the airgap and create retention locked copies. Finally, a “Configuration and Recovery Procedures Guide”, specific to the environment, is developed to define recovery procedures, from which a recovery runbook can be developed.

While the IBM i is a platform that can provide unparalleled security, the current threat environment requires that companies develop and understand a process for protecting and recovering critical data in the event of a malware attack. As infrastructure architecture evolves, threat protection must evolve to keep pace with emerging threat vectors and the security implications of providing access to critical resources.

Who is Entrepid?

Entrepid is a technology firm specializing in IBM Power Systems infrastructure and integration with storage and backup technologies.

Our practice originated over 20 years ago providing host and SAN integration expertise for integrating IBM i and EMC Symmetrix. We have since contributed to and incorporated advancements in the platform, modernizing our clients’ environments – many Fortune 500 and Global 2000 companies among them. From backup and disaster recovery and custom automation to preconfigured, converged solutions for IBM i and other platforms, Entrepid helps clients make an intelligent transition from legacy to modern technology architecture.

This content was sponsored by Entrepid.

Brian Barth is president at Entrepid.