Author(s): Prashanth Kodurupati
IBM Sterling Integrator supports a wide range of file transfer and communication protocols, including SFTP, Connect: Direct, and Mailbox (HTTP). While connecting with client servers, it’s imperative that these protocols and all other prerequisites for a successful file transfer, including authentication protocols, encryption, and file naming conventions, match between the IBM Sterling adapter for a connection and the client server.
With a significant market share across multiple verticals, IBM’s Sterling Integrator is one of the most widely used B2B integration platforms with robust bulk communication and data exchange capabilities. It facilitates secure connectivity between businesses and their B2B clients and data transfer and streamlines the process through various features. However, for a seamless transfer to occur between a business and its client’s servers, the configuration has to align for each connection, especially when it comes to data transfer and communication protocols. Three of the protocols Sterling Integrator natively supports are SFTP, Connect: Direct, and Mailbox (HTTP).
There is ample literature on two of these three protocols and relatively limited (and mostly from a single source - IBM) literature on Connect: Direct, which is a proprietary file transfer solution. In both cases, the literature covers a comprehensive range of scenarios, including the challenges that may arise if there is a protocol discrepancy. Secure File Transfer Protocol (SFTP) allows the transfer of files/data over a network and is among the most widely used secure file transfer protocols, capable of catering to a wide range of use cases, including remote systems, but they need to be configured as per the requirements of the connection and transfer [1]. Most mailboxes, whether they are operating in conjunction with IBM Sterling integrator (or under it), rely on HTTP for their access layer, but there are other options as well, and even HTTP-based mailboxes may vary greatly based on their inherent security and other features [2]. Literature on IBM’s proprietary Connect:Direct file transfer mechanism (or protocol) focuses on its strengths, typically in the context of other IBM systems, and the official IBM documentation on the topic covers everything from its features to configuration [3]. There is also ample literature, mostly from IBM and other resources, on how IBM Sterling should be configured to ensure secure and seamless data transfer [4].
Before evaluating the problem itself, we have to consider three of the commonly used communication and data transfer protocols supported by IBM Sterling integrator that are at the root of this problem.
It’s important to understand that IBM Sterling supports a wide range of protocols, but based on use cases, industries, and preferences, businesses may only work with a limited set of these protocols during their operations.
Secure File Transfer Protocol (SFTP) is a variation of widely used File Transfer Protocol (FTP) and is built on top of Security Shell (SSH), an encryption protocol. This allows SFTP to secure both data and instructions being exchanged between the client and host servers via encryption. It’s also quite versatile.
Connect: Direct is a proprietary peer-to-peer integration middleware that facilitates secure data transfer, both high volume and massive file size. It also allows for automation, making it an ideal tool for a wide range of Managed File Transfer (MFT) scenarios.
HTTP-based mailboxes, whether used in an IBM Sterling environment or with another integrator, lean more toward communication than data transfer.
The host server leveraging IBM Sterling integrator has to treat each connection for file transfer and communication individually/ separately, and if it’s misconfigured, messages and data won’t transfer between host and client servers (either way) and will result in an error side requesting the transfer. This can prevent client servers from requesting important and, in some cases, missioncritical data from host servers.
The overarching solution to this problem is to configure each connection as per the client’s requirements, for which the first prerequisite is setting up the connection using the same protocol that the client-server will use to transfer the files. For example, if they use SFTP, the connection should be configured for SFTP (an SFTP adapter) [5].
However, there are other considerations beyond matching the file transfer/communication protocol. This includes authentication protocols that the client is using as well as their encryption protocols. An encryption mismatch and the presence of incompatible encryption protocols on either side of the connection can prevent files from being transferred successfully between client and host servers.
When setting up a more advanced solution/protocol like Connect:Direct, there may be additional requirements, like setting up the right node configurations. Similarly, a different set of configuration requirements may apply when setting up a mailbox within the IBM Sterling integrator environment, including choosing between NIST or FIPS compliance based on the client’s requirements.
In conclusion, the file transfer protocol configurations have to take into account the client’s requirements and protocols, and while configurations may be swapped between clients with similar requirements, a universal set of configurations governing all file transfers and connections will not be advisable.
The following scenarios and solutions go beyond the protocol level mismatch.
The most important practice when establishing connections with new client servers is to obtain all the necessary information required to properly configure a client-host server connection, including the file transfer/communication and authentication protocols they use. Access to this information at the time of configuration can prevent failures and reconfiguration.
FTP engineers and other professionals responsible for ensuring IBM Sterling on the host side and creating adapters for client connections should look into a broader set of mismatches than just configurations. This includes file naming conventions. If you have a different naming convention than your clients, it may cause transfer errors, or even if the transfer is successful, there may be traceability and usability issues.
Creating configuration templates for some of the most commonly used adapter settings and connections can prevent repetitions when setting up connections for new clients.
The solutions can be implemented in every host-client connection scenario where the host uses IBM Sterling to connect with their clients/client servers and govern data transfers. However, the solutions are limited to three primary file transfer/communication protocols, i.e., SFTP, Connect: Direct, and Mailbox (HTTP), and if the clients are using protocols other than that, they may require a different set of solutions. However, the underlying aim in almost every scenario would be to match host-side adapter configurations with the client requirements.
File transfer and communication protocols are a core element of every secure connection and file transfer instance, and any clash among these protocols between two server entities initiating a file transfer or connection can lead to a failed file transfer. However, if it’s identified when the file transfer is critical to business interactions/operations, it can result in financial and reputational losses. Therefore, the goal should be to have every client connection in IBM Sterling tested and configured correctly.
