How Can I Trust My CRM?

A well-known, global, luxury retailer asked us this question. We looked at more than 11-million of their customer records and discovered that nearly half of all the customer records (45%) contained bad data.

• 43% did not have data in the country field.
• 38% had unusable terms or phrases, such as “Needs New Address,” in the data
• 12% had duplicate information in different fields, such as the same entry in the ADDRESS 2 and CITY fields.
• 5.5% had identical values in the given name field and the surname field.

More details and examples may be found in our report Quantifying Data Quality Errors.

How Big Is My Data Problem?

(back to top)

You can reduce the amount of bad data right at the source by tackling the three primary causes of bad data in CRM and CDP systems:

1. The customer data wasn’t entered consistently or accurately.
2. The customer data has changed and wasn’t updated.
3. The customer data was improperly integrated from different systems that were not designed for the same purpose.

Focus on the following areas to minimize these three causes of bad data:

Customer Form Design

Make sure that your customer and sales representatives know what to do. Many form designers think that their instructions and layout are clear. But, we are dealing in a world with cultural and physical differences that need to be handled.

One pervasive and easily fixed problem has to do with instructions for name entry. It is common in Western Europe and the Americas to ask for a person’s “First Name” and “Last Name.” However, in Japan, Korea, and Hungary the family name is used first.  Solve the problem by using terms like “Given Name” and “Family Name.”  (Personal Names Around The World.)

The other common problem is that form designers tend to create forms based on the structure of the database rather than the way people want to enter data.  How many address lines does your customer need to ensure delivery?  How does she enter a post office box, her tower number, her home’s name?  It may be best to have large open fields that can be parsed by software into their proper components.  If a form structure is required, it is essential to have clear instructions. (More about addresses in forms may be found in an interesting article entitled Falsehoods Programmers Believe About Addresses.)

Data Entry Mistakes

Reduce typos by looking up the customer in real-time and through the use of autocomplete software.  You might be able to retrieve customer information by asking a customer for her phone number first. The entire form might be completed automatically, which reduces the chance of a typo. The customer can review and make corrections for her on the spot.

In the absence of real-time customer information, real-time autocomplete works well by suggesting addresses as the person types. After she types a few letters, entire addresses can be displayed on a list from which she may select.  It saves time and reduces errors. If real-time autocomplete is not feasible, specialized software can also check her entry after it is completed and suggest corrections based on a database of address information.

Proper Data Silo Integration

Luxury retailers have many data silos: online and brick-and-mortar, country by country, and product lines. Each was created for a different purpose and likely has different data in a differing format. When data from these silos is merged, new errors are produced. A complete discussion may be found at Data Silos.

(back to top)

For the data that already exists in a database, data accuracy can be improved significantly with address cleansing, name matching, merge/purge, and suppression.

Consistency is a critical first step.  Address cleansing and standardization is a record-by-record process that involves removing common errors, adjusting address formats where appropriate, and then verifying each address. Automatic fuzzy error correction corrects switched letters, missing words and letters, standardizes abbreviations, and handles missing or extra spaces.  Phrases like “Needs New Address” or “Unknown” can be flagged. It is essential to make sure that country, language, and regional issues are top of mind when cleansing and standardizing as the patterns can be different in significant ways.

A similar process can be used to verify email addresses and phone numbers.

Once all of your data is cleansed and standardized, you can start the process of matching across customer records to increase accuracy and completeness further.  Standardized records for the same person may not match exactly. Personal data is always changing. People move. Individuals may provide a work phone number during one interaction and a mobile number during another interaction. Many people also have multiple addresses (e.g., primary and vacation homes), multiple phone numbers (e.g., fixed landline and mobile), and numerous email addresses (e.g., personal and work).

Matching records with these subtle issues can be challenging.  A technique known as “cascading” assembles data from different cleansed records for the same customer.  It looks for differences between the records and uses advanced logic to resolve conflicts and fill in incomplete information.

The ideal solution is not to change the source document, but to create a system of reference that can be synchronized with the various data silos.

(back to top)