Building Robust CRM Integrations with AWS Step Functions

Introduction

Large, digital-first organizations rely on a wide variety of tools and platforms to manage marketing, financial processing, and record tracking. This results in a critical need to integrate platforms to ensure data parity and accuracy across different systems.

Customer Challenges

Unfortunately, our customers frequently discover that the no-code integration options provided by their SaaS platforms are inadequate for their needs. Heavily marketed "native integrations" are rarely as comprehensive as advertised. Zapier integrations are limited by the target platform's exposed triggers and actions; they become expensive and unwieldy at scale, frequently requiring developer support and lacking the maturity to handle errors and preserve data.

The challenge for this non-profit organization was to create a low-code, robust systems integration to connect Stripe donation processing with a custom donor portal, Virtuous' Donor Management Platform, Tatango's SMS Marketing Platform, and HubSpot's CRM.

Here are a few of the larger challenges and goals we tackled as part of this project:

• Move the client off of Zapier to reduce cost and complexity
• Work around restrictive Virtuous and Tatango API rate limits
• Overcome the limitations of Tatango's one-way, native HubSpot integration
• Overcome the limitations of Virtuous' native Hubspot integration that only syncs certain key data fields one-way
• Address the Virtuous Import Tool's drawbacks that introduce a 24-hour time delay for all posted contacts and donations
• Off-set donations for end-of-year receipt flows based on donors' local timezones
• Address the need to catch errors quickly and retry failures programmatically to minimize cost and ensure data consistency.

The Solution

We delivered a scalable, flexible, manageable, and cost-effective solution by implementing a “serverless” architecture (only “on” when in use) instead of an always-on system that requires on-going maintenance and cost even when not in use.

We laid the foundation for further iterative improvement projects related to data processing and management.

Customer Success

Thanks to our multi-environment testing and deployment processes with AWS SAM, our customer was able to move to their desired platforms with no data loss and minimal downtime.

By moving off of Zapier and expanding on native platform integrations, they gained visibility into the performance of their integrations, allowing quick resolutions for any issues that arise from a technical, security, or donor communications perspective.

Our customer has the ability to recover in the event of rate-limit errors without data loss and without developer input. Unlike Zapier or other platform integrations that are not controlled in-house, these low-code integrations prevent silent failures and provide retry options so that no data is lost when unexpected problems arise. Our customer can be certain that their donor information is complete and accurate across all supported systems.

Other benefits of this low-code solution include...

• Cost savings - Our customer was able to reduce their Zapier plan from $2500/mo to below $600/mo
• 200K/mo reduction in Zap runs
• End-of-year offsets for donation receipts visible in all sources of record, so that donations are placed in the correct tax year for the donor's local timezone.
• Integrations that meet our customer's needs by ensuring data syncs both directions and fields that can be mapped as desired.