White Paper
Message Systems Delivery Manager
Introduction
Companies devote significant resources and dollars to setting up and maintaining IT infrastructures to keep their ever-increasing outbound email communications flowing. Moreover, marketing commits hours to developing messages to motivate prospects and customers. And, yet, despite best efforts far too many of these finely crafted and highly critical messages never make it to the intended recipients' inboxes. The simple truth is that while email sending technologies have dramatically improved in recent years, getting messages to the inbox is harder than it ever was.
The reason is clear: businesses and email service providers face a variety of evolving challenges created by the escalating volumes of spam, spoofing and phishing messages generated by illegitimate senders. In an effort to shield their customers from these intrusive and unwanted messages, ISPs are implementing aggressive countermeasures, including domain signing, throttling, blacklisting, tar-pitting, spam filtering and relationship management. While necessary, these prevention techniques are not perfect and often result in blocking legitimate messages. This constantly-evolving environment makes it increasingly expensive and difficult for reputable senders to get their email into their prospects' and customers' inboxes. According to Jupiter Research , undelivered email cost businesses and email service providers almost $110 million dollars in 2006.
To ensure their mail gets delivered, legitimate senders must develop sound strategies to keep pace with the ISPs' changing requirements on how they want to receive mail. Moreover, senders must be able to economically scale their IT infrastructure to handle increasing volumes of messages in a high-availability environment.
This white paper presents the issues senders must overcome to achieve optimal deliverability and describes how the Message Systems email software solution can help reputable corporations and email service providers (ESPs) meet these challenges, while maintaining an infrastructure capable of scaling to handle any volume of messages.
Deliverability Challenges
Poor deliverability, list management and reputation management as well as ISP anti-spam efforts and the inability of common message transfer agents (MTAs) to manage and scale are just a few of the challenges senders face in their quest to deliver legitimate messages to their prospects' and customers' inboxes.
Email Deliverability Speed
As the volume and size of a sender's messages increase, performance is often compromised. The traditional IT infrastructures-including open source solutions-used by most senders are not capable of scaling to handle the increased mailing requirements at desired performance levels.
To support rising mail volumes, senders typically implement multiple instances of a MTA on additional servers to augment their existing IT infrastructure. While appearing to be a logical solution to this performance problem, most companies soon learn that the resources to sustain and maintain this type of solution eventually outweigh the benefits.
Bounce Processing / List Maintenance
Rejected messages represent lost opportunities and wasted resources. While most senders can cite statistics on how many of their messages are undelivered, few know why. Furthermore, many systems only handle bounces that occur during delivery, ignoring the out-of-bound rejection messages that arrive hours or days after the delivery has occurred. Knowing about every bounced message as well as why and by whom a message was rejected is critical information that enables senders to take immediate corrective action to boost future deliverability.
To compound the problem, ISPs do not use standard status messages and codes in bounce messages. What's worse is bounce messages from the same campaign can vary based on dynamic content such as hashes and per-message identifiers. Without some method for categorization senders may be spinning their wheels interpreting duplicate statistics. Traditional email solutions do not provide this type of data collection and categorization. What's more, many systems still require human intervention to convert a hard bounce into a mailing list update. A solution that can automatically capture and organize both in-band and out-of-band bounces provides the information necessary to effortlessly manage mailing lists.
ISP Throttling
Many ISPs have incorporated throttling and tar-pitting (adding a delay between e-mail messages) to help reduce the flow of illegitimate messages to their customers. To be compliant with these practices and ensure their legitimate messages make it through, senders must throttle their message delivery to match each ISP's throttling requirement across the entire email platform. ISP set limitations and senders must adhere to these rules to be successful.
Throttling also impacts the sender's infrastructure. When message delivery is delayed, the infrastructure must be capable of efficiently queuing hundreds of thousands-or even millions-of messages for future delivery attempts without bogging down performance.
Reputation Management
Most ISPs have systems in place to individually scan incoming messages for viruses and spam, check them against blacklists, and evaluate them with any number of other attributes. Reputation is one of the criteria ISPs are using to evaluate mail. ISPs do this by requesting the sender's reputation score from a central third-party reputation database.
Reputation management promises to significantly decrease the amount of spam received by users. However, it can add a layer of complexity to the sender's infrastructure if not implemented properly. Senders who already have an email solution that can readily accept new standards will be able to easily adapt to reputation management practices. Those with less flexible infrastructures will need to upgrade their existing MTAs or add gateway products, which can add a level of complexity and compromise performance.
Cumbersome MTA Management
As senders work to adapt their sending practices to match ISP requirements, configuring legacy systems to implement new messaging standards can be challenging at best and expensive and ineffective at worst, often leading to configuration errors, wasted time and lost opportunities.
For example, some systems are so difficult to change that they force the use of cryptic configuration files and require a restart-resulting in service interruption-with every configuration modification. IT resources are especially strained in infrastructures that require multiple servers to handle increased loads because each machine must be individually configured each time a change is required. This is especially time-consuming and prone to inconsistencies if the IT staff lack central management capabilities.
Growth Limitations
As a company becomes more successful, the size of its mailing list increases through the addition of new customers and prospects, further stressing the messaging infrastructure. To handle the escalating load, IT must find a way either to scale up or out, while maintaining effective resource and cluster management.
To ensure maximum flexibility and performance, senders also need to integrate new standards and processes into their MTA. This requires tools with extensive programming hooks and well-defined APIs that allow for the efficient integration of legacy systems and business logic with mailing solutions. Senders that have an inflexible mailing system infrastructure that lacks these integration tools will generally spend more time and money to replace or upgrade their infrastructure to support the new technologies that are essential for ensuring deliverability.
The Message Systems Solution
Message Systems Delivery Manager was designed to help senders overcome these common deliverability challenges in a straight-forward and economical way. An enterprise-class mail transfer agent, it was built from the ground-up for performance, flexibility and extensibility.
Intelligent Queuing Systems
Message Systems Delivery Manager uses an intelligent queuing system to solve the speed-of-delivery problem caused by increased mail file sizes and volume regardless of queue size. Using individual queues on a per-domain-per-basis, it provides the best-possible queue management at granular level. Senders benefit because slowdowns for a given domain on a given IP do not slow down messages on another domain/binding.
In addition to the highest queue granularity in the industry, Message Systems Delivery Manager also employs the industry's most advanced queuing algorithms to ensure delivery does not slow as the queue grows under real-world loads. Using these queuing algorithms, the time needed to retrieve a message from the queue stays the same whether the queue contains one message or millions of messages.
Advanced Bounce Processing
With the ability to capture, analyze and interpret in-band and out-of-band bounce messages, Message Systems Delivery Manager provides superior bounce processing out of the box. In-band bounce messages can be analyzed and forwarded to the original sender as out-of-band bounce messages. Out-of-band bounce messages can be analyzed and then be either forwarded to the original sender or dropped.
Message Systems Delivery Manager also provides extensive programming hooks and APIs so that senders can integrate existing bounce processing systems directly into the sending infrastructure at the MTA level. To aid in analysis and increase future deliverability, Message Systems Delivery Manager performs the categorization and classification necessary to provide accurate metrics. The Message Systems web console displays deliverability statistics, including a list of bounces correlated to the appropriate ISP.
List Maintenance - In addition to providing metrics for users to identify specific delivery issues, Message Systems Delivery Manager can be used to automatically update and maintain mailing lists through its well-defined APIs and database interfaces. For example, email addresses that suffered a hard bounce-a message that is rejected because the recipient address is no longer available-will be automatically deleted. The ability to have the mailing list automatically updated reduces the hours needed to manually cull lists and conserves sending resources by limiting the number of outbound messages to only those with valid addresses.
Accreditation and Reputation - Message Systems Delivery Manager can be integrated with mailbox monitoring seedlists provided by third-party providers such as Habeas and ReturnPath to provide a higher level of delivery detail. By pulling from a network of special accounts from all ISPs used, seedlist integration provides details on actual inbox delivery rates, including the final disposition of each message.
Message Systems also supports Goodmail CertifiedEmail messages. CertifiedEmail is a trusted class of email that assures deliverability and automatically renders links and images for accredited senders through best sending practices. This technology is currently in the process of being deployed by seven major ISPs.
Granular Message and Connection Throttling
One way for a sender to maintain its good reputation is to control the speed of message delivery to avoid overburdening ISPs that have implemented throttling. Message Systems Delivery Manager provides the granularity necessary to comply with individual ISP throttling requirements by giving senders control over settings such as total outbound connections, total message volume and volume ramping. These controls can be set on a per-receiving domain, per-MultiVIP binding and per-MultiVIP/per-domain basis.
Message Systems also supports multiple virtual outbound IP addresses through its MultiVIP® architecture, which allows groups of messages to be sent from a unique outbound IP address.
One of the advantages of granular throttling and separate queues is the ability for senders to assign different levels of service to different customers or types of mail, depending on the importance and urgency of the message. For example, this can be used to provide priority transactional messages with a higher quality of service than bulk messages: one customer could pay for 100,000 messages per hour, while another could pay for 500,000 messages per hour.
Clustered Logging, Clustered Configuration and High Availability
Message Systems is the only email solution provider to offer full clustering-configuration, logging and metrics -with high availability. A clustered configuration has several advantages. It makes server configuration and management easier and less costly, while ensuring all servers are simultaneously configured the same. It also provides automatic failover to ensure delivery continuity and it provides an easy way to retrieve data from servers for timely reporting and analysis.
A clustered IT infrastructure enables senders to create a master configuration on the cluster manager that is then pushed to all nodes in the cluster. This updates each server at the same time to ensure configuration consistency. For exceptions, automatic overrides can be created and acquired at startup for individual nodes.
Clustered metrics allow an administrator to enforce quality of service across an entire cluster instead of on a per-node basis. For example, without clustering a sender would have to divide the limit by the number of nodes to enforce a connection limit to a specific domain. Conversely, using the Message Systems Cluster Manager within the Delivery Manager, senders can set connection limits for the cluster as a whole and individual nodes will take each other into account to maintain the cluster-wide limit.
The high availability provided by a clustered Message Systems solution is vital for senders that need to maintain service levels in a mission-critical environment. The high-availability feature provides dynamic node failover in the event that a server fails or is taken offline. Even if multiple nodes fail, the remaining servers will immediately take over-handling the incoming IP addresses and queues of the failed nodes-to ensure continued operation.
To aid in analysis, the Message Systems JLog clustered logging architecture aggregates and displays logs from each cluster node to a central cluster manager, providing valuable deliverability statistics on demand instead of waiting days to manually collect and compile the data. This timely information can then be used to make adjustments that will have an immediate positive impact on the deliverability success rate of subsequent mailings.
Injection and Management APIs
Message Systems Delivery Manager fully integrates into a sender's infrastructure and accepts business rules through a series of programming hooks and APIs. This enables senders to use APIs to manage message validation and logging, and server management.
Senders can also use Message Systems Delivery Manager's numerous programming hooks to inject business logic to customize every stage of the message delivery process. The Message Systems injection API allows senders to achieve a higher throughput than is possible using regular SMTP or ESMTP protocols by removing communications overhead and allowing messages to be transferred using more efficient binary protocols.
Conclusion
For companies and organizations that depend heavily on email communications to market and conduct business transactions, deliverability to intended inboxes is paramount. Unfortunately, many senders find acceptable delivery rates elusive.
This can be caused by a combination of factors, including an outdated IT infrastructure that does not scale easily to handle increasing volumes of mail or support new sending standards; the difficulty in obtaining useful information about why a message bounced so that lists can be updated before subsequent mailings go out; and the inability to achieve and maintain high levels of throughput. Moreover, as ISPs introduce new programs and technologies to prevent illegal messages from reaching their customers, reputable companies must be able to adapt by quickly implementing sending strategies to comply with new requirements.
Message Systems Delivery Manager solves these problems by providing flexible sending solutions and services that address the email infrastructure and deliverability needs for organizations of all sizes to ensure deliverability now and well into the future.