How to Fix Data Disasters in Marketo

This is a somewhat inadvertent follow up to this week’s Six Principles of Data Cleanliness.

I was working with a client and for some reason we decided to setup an “Unqualified Auto Delete” flow. At the time, it seemed like a good idea because they did not want to keep disqualified leads anymore. It looked something like this:

Member of Smart List IS "Unqualified"
Delete Lead, From CRM IS TRUE

Bonus points if you can tell me why this is bad before we continue. The campaign was set to run nightly, on a daily basis.

Here’s what it should look like (and I don’t recommend this most of the time):

Marketo Auto Delete Flow

In the meantime, I went in and built a lead lifecycle system, with the brilliant idea of an Auto Disqualify flow. This would take all of the early stage leads that they never wanted and just set Lead Status=Disqualified. Naturally, the Unqualified Auto Delete would handle the rest. How many leads would this affect? Not too many, right?

Wrong.

A bit of a logic error led to a whole ton of leads being pushed to Disqualified and into the waiting jaws of the Unqualified Auto Delete campaign.

<unmentionable words>

Except we were saved by a weird Marketo error and my own foresight. First, I had noticed the Auto Delete did not have a Wait Step. I set that for 30 days before I completed the lead lifecycle. Second, the Queue names had changed in SFDC, but somehow Marketo never updated them in the Change Owner step that predated the name change. So when the Lead was Disqualified, it was never moved from its current owner. (phew!)

But the Scores were changed and that was not as pretty. So now I had 4000 Leads waiting to be deleted from both systems, all without Scores, in the wrong Lifecycle Stage, and with the wrong Status.

How did I roll this back? These lessons will be important if you want to correct the inevitable data error.

Step 1: Who was affected?

This part was fairly easy. It turned out that the Auto Disqualifier was the culprit, so anyone who was a Member of Campaign IS “Auto Disqualifier” is our target. You can also look at Campaign Members at the top of each flow.

Turn that off immediately.

Step 2: Remove from Flow – Save the Leads!

Much like the liver, we better go save those leads from Auto Deletion. In this case, we probably only needed to turn off the Schedule Recurrence until we cleared up the mess. Just in case, however, I ran a Remove from Flow on all the Leads. This is a useful flow step that has been much neglected since the demise of the traffic cop.

remove-from-flow

It’s a good idea to turn off any flows that may affect these leads or other leads.

Step 3: Reset Data Values

I looked at the Lead Statuses and set everyone who was “Disqualified” to “Open”. It’s not easy to go back unless you had been tracking field history on Lead Status in SFDC. Most firms do run history on this field, so make sure you start to do this.

Then we have to move everyone back from Stage=Disqualifed to Stage = ??. In this case, the Auto Disqualifier was only set to listen for Engaged Leads. Unsurprisingly, as soon as the remaining leads were pushed to Engaged, the Auto Disqualifier did its work. So I just set everyone back to Engaged.

Step 4: Restore Scores with a List Import Overwrite

In Marketo, as you might know, it is really difficult to roll back a data change unless you happened to know precisely what occurred. Thus, Step 3 was easy because we knew all the Leads were Engaged or Disqualified. With Scores, however, we have no idea what it used to be.

Fortunately, another quirk of this system is that all the Score fields were being tracked by SFDC history. I was given a spreadsheet of Old and New Values and worked to get the sheet ready for an import. Once I had the fields:

Lead Score | Behavior Score | Email Address

I could proceed with an import. Marketo will match the records and overwrite the existing scores. Keep in mind that some scores will be above the MQL threshold, thus the Data Value Change will trigger the MQL stage. That is ok, but be prepared for hundreds of leads being pushed to Sales. Warn someone!

If you are field blocking one of the fields you are trying to overwrite on import, Marketo will let you know. Check Admin > Field Management and temporarily remove the restriction.

Step 5: Ask Why This Happened

By this step, I already knew. I had failed to do enough quality checking and testing due to time constraints on the project. Always give yourself enough time to include full testing on a system as complex as Lead Lifecycle (LLC).

Of course, correct any flows that you can.

Step 6: Switch it Back On

After the cleanup was done, we turned the corrected flows back on. The exception was the Auto Disqualifier. I did not trust it to do its job even though I fixed much of the logic. It was too risky.

Remember these key steps to correcting your data errors:

  1. Who is affected?
  2. Turn off campaigns and remove from flows
  3. Reset the data values
  4. Use a List Import to reset data values (optional)
  5. Ask why this happened
  6. Switch corrected flows back on
  7. Call Marketo Support (if you can’t figure it out).

And here are a few tips on setting up your system to help you repair errors:

  • Never automate deletion of leads without at least a 30 day wait step. (I don’t recommend this at all actually, you should do this as part of your Marketo housekeeping).
  • Turn on field history in Salesforce for key fields like Score, Lead Status, Lifecycle, etc.
  • Test all of your flows.
  • Make sure SFDC is backed up.

Have you had any mini disasters with Marketo? How did you solve them?

The Six Principles of Data Cleanliness for Marketing Automation

This post expands on my post at RingLead.com: Six Marketing Automation Principles to Keep Your Data Clean.

Do Not DuplicateMany marketers who are new to marketing automation suddenly discover their success on the job is directly affected by databases that seem out of their control. CRMs have data that is imported by salespeople, interns, hand typed, etc. That one hot shot sales director you poached from your competitor? He brought his “personal” list over and uploaded 5,000 records with no Lead Source, few Emails, no unsubscribe, no lead assignments, and uploaded it all without permission.

Sound familiar?

Almost always this data is not imported correctly or with consistency. It is just there!

But you can take control. If you want your job to be successful with marketing automation, you must take control – and take responsibility – for the data from Sales. (and sometimes your CRM managers). You need not worry, you can do this and do it well with the principles of data cleanliness.

Principle One: Design the System to Avoid Human Error

Marketers I have worked with have a familiar refrain: “Sales, Leads, and Other People are always importing bad data into the CRM.” When I look at their CRM and Forms, the most common root cause of these errors is the design of the system. Simply design your workflows and access levels to encourage certain behaviors while blocking bad behaviors. This is the first principle because the others rely on system design.

How do you design the system to make behavior work for you? Let’s try a few common CRM tools at your finger tips.

Picklists

Picklists ensure that your field values are completely static and controlled by you. The user or lead gets a choice, but the spellings are always precise. Common fields to use these on are: Country, State, Lead Source, and Lead Status. For countries and States, I use the ISO Country List.

Read Only Fields

Some fields are meant for information to Sales or to Marketing. Thus, only a few people should ever be able to access the field. For example:

  • Comments History
  • Interesting Moments
  • SFDC Campaign Member
  • SFDC Campaign
  • Lead Source
  • Subscription management fields

Import access must be limited to certain people

Even within Marketo, you may want to restrict junior or inexperienced staff from importing lists. The most well intentioned user can create havoc by not following procedures. With a marketing automation platform (MAP), always import into the MAP, never into the CRM. Of course there could be exceptions, but only an administrator would really know when those should occur.

Role vs. Title

Another design principle is to use picklist fields and free text fields at different times. For example, Role is a picklist that you might put on a form early in the buying process. Later, or in a progressive form, the Title field would appear. The reason I like this two-part data collector is that Role is easy to just choose. Titles vary widely and are for most people’s vanity. Lead scoring and other routing based on free text fields has to take into account a lot of options. Why not avoid issues early on?

Principle Two: One Field for One Purpose

The field that comes to mind is Description (in SFDC) or “Notes”. This field was intended to be for either the Lead to fill out some detail or to provide a scratch space for Sales. Instead, it is often used to put in call notes and the history of the Lead.

Do NOT permit this to happen. Sales absolutely must use Tasks or Meetings to log calls, even voicemails or emails. These records are stamped with time and date and have appropriate fields to help track Sales’ interaction with the Lead. Tasks are also a great way to tell Sales what Marketing is up to.

Thus, a field like “Description” is best used as a temporary field for handling data that will be passed to a Task or a “Comments History Field” [see instructions].

Principle Three: Use Automation to Correct Data Values

The use of automation rules to correct common misspellings or missing data goes back to the 1960s and early principles of data quality when direct mail errors could result in thousands of dollars wasted. Today, data quality mistakes ruin reputations far faster than a misdirected mail piece. Email reputation, choice of language, and customer experience matter and they matter more because mistakes can go publicly viral in minutes. Save your firm’s reputation with automated data cleansing.

These data management flows are one of the first things I setup in Marketo. Most firms setup these flows to start.

  • Country Corrector (although you should have followed Principle 1 to not need this)
  • State Corrector or Mapper – often helpful if you have Country and Inferred State.
  • Count of Employees to Employee Range – again, Principle 1 and 2 say you shouldn’t need to do this, because you chose one option, but it’s possible you ignored me :)
  • Bad Lead Source to Good Lead Source
  • Email Invalid to Email is Good if the Email changed.

For example, this flow adjusts Employee Range based on Employee Number.

Old Employee Range Corrector

Principle Four: Prevent Duplicates at the Source

Your system can include tools from the Appexchange such as Dupeblocker or RingLead to stop a user from entering a duplicate at that very moment. If you get the settings right, this should discourage salespeople from just entering data without thinking.

Then you should run a deduping process over your entire CRM before you connect your marketing automation platform (MAP). You will thank me later. Most marketing automation firms offer a cleansing service (including Marketo), but you can ask other firms for help. If you have an SFDC Admin, they will know the tools and services to help you dedupe effectively. Usually this involves DemandTools, if your company can afford it.

Principle Five: Establish a Regular Cleaning Process

Remove DuplicatesThis cleaning process includes a monthly or quarterly review of the MAP+CRM. You will want to bring your team into a room, with laptops, and with pizza. Spend an entire Friday afternoon doing this and life will be better the rest of the time. Assign one person to each area at the start, and then go from there.

  • Duplicate count review: find out who is doing this and clean it up.
  • Asset Cleanup: turn off or archive old emails, landing pages, workflows, programs. Marketo does a great job of allowing archiving to retain data. Also, team members should delete tests and other items they no longer need.
  • Workflow Review: are the right ones working properly?
  • Empty Value Counts: review records that are still missing key data: Country, Employees, Annual Revenue, Industry, etc… whatever is crucial to determining customer fit, scoring, and MQL status.

Principle Six: Automate Data Appending

You will always have leads without key demographic data points. These leads could be interested, but if you do not know they are in your target audience, you will never pass them to Sales. Nor should you!

Instead, find an appropriate data appending service. There are two types:

  • Form Based Autofill with ReachForce, for example. These tools ask the Lead to confirm if they are at a specific firm. The Lead confirms and the system autofills geographic and firmographic data automatically in the background.
  • Backend Autofill: these services are similar to Hoover’s, DiscoverOrg, RingLead, and ZoomInfo. Their integration or service backfills missing firmographic data based on your rules. This happens in the background, but the Lead never sees this.

Often, firms use both front end and back end appending tools to cover as much of the data gap as possible. Whichever one you prefer will help you improve segmentation capabilities and reduce the number of unaddressable Leads.

Remember these principles as you clean your database:

  1. Design the System to Avoid Human Error
  2. One field is for One Purpose
  3. Automate Data Correction
  4. Prevent Duplicates at the Source
  5. Establish a Regular Cleaning Process
  6. Automate Data Appending

Here are a few resources you can use for preparing your database to be clean in and out:

New to marketing automation? Learn more about MAP administration strategy with my webinar with RingLead. View Now.

(Disclosure: I sometimes write for RingLead).

Image Credits: Whatleydude, Barbourians