Meaningless Surveys

In Customer Service by J Michel Metz2 Comments

We were almost done. We had just agreed upon a price – finally – and were looking forward to getting the hell out of there.

The salesman approached with some paperwork, including a survey that Ford was going to send me to rate the experience. Up until that point in time there had been some awkward moments, but nothing that was too serious. Then he started talking about the survey.

So pretty, but what a pain buying.

By this point in the day, I was tired of putting up with the typical car salesman garbage. There is a very, very good reason why people dislike car salesmen, and this was a masterclass day of significant levels of bullshit. Forget the hip waders, you needed a hazmat suit.

The last thing you want me to do right now is fill out a survey.

No Good Moods

I did not want to be there. I did not want a new truck (well, I did, but not like this). I was angry at my situation, upset at my need to spend money, upset at having to take time out of my day to deal with these assholes. I just wasn’t in the right frame of mind.

The decision was taken out of my hands, of course, because of an unfortunate accident that had occurred earlier in the week. My 2007 XTerra wasn’t the best car, or the most fun to drive, but it was 100% reliable and 100% paid for.

One of the dangers of living in rural areas is that you get too many varmints that will dart out in front of you. Fortunately, aside from some bruised ribs and some issues with my left eye, I came out otherwise unscathed. Sadly, the XTerra was totaled.

All of this meant, though, that I was in pain (bruised ribs will really, really put a damper on your day), cranky, and my tolerance level for crap was about a fifth of what I can normally take.

Enter the car sales people.

Car Buying (No-) Service

I tried to make my life easier by going through Navy Federal’s Car Buying service. Apparently some people had some good experience with theirs and I have always had a good relationship with Navy Federal (I’ve been a member there since I was 11 years old, back when a savings account was recorded in an actual, physical book). I’ve always gone through Navy Federal, and I wouldn’t think of going through anyone else for a car loan.

Like many banks and credit unions, Navy Federal has a car-buying service. If you’ve never used a car buying service before, what is supposed to happen is that the financial institution is supposed to have a certain group of car dealers that they work with to get the ‘best price’ for any given vehicle. The car dealer is supposed to show you the manufacturer’s invoice and what the savings is going to be. There’s not supposed to be any negotiation, because it’s a ‘no-haggle’ approach.

Supposed to.

When I went to the dealership, though, it was as if they had never even heard of such a thing. I mentioned Navy Federal several times to the sales person (and even the sales manager), but it was expertly brushed off. When I had found the truck that I was looking to buy, I asked for the Navy Federal price. The sales guy went into the back, and then came back with… dealership financing.

No invoice. No Navy Federal price. Dealership pricing.

“I know you said that you weren’t going to finance,” he said, “but I have to go through this.”

Why? Why do you have to go through this? As he pointed out the different monthly options, rates, and the like, I kept looking at the price of the truck in small print (the financing numbers were in a font at least 2x-3x the actual size of the price amount).

I redirected his attention back to the price of the truck. “I can’t do this,” I said.

“Yeah, but if you finance with us, I can take another $750 off in incentives.”

“I’m not financing the truck. I’m going through Navy Federal.”

“I’ll be right back.”

Away he went into the back, and once more I hoped that he would come back with the proper paperwork. Alas, I was disappointed once more. This time, though, the financing numbers were bigger, but so was the price of the truck. That is, they had increased the price of the truck.

After going back and forth on this for a third time, I stopped him mid-sentence. “Look, I know you have to do this,” I said, trying to keep my voice level. “But I’m starting to get really irritated. I’ve already told you my situation, I’ve already explained I’m not financing this through the dealership. You are supposed to have something with Navy Federal, and it’s the only reason why I walked in the door in the first place.”

“Don’t go anywhere,” he said, realizing he was on very thin ice. “I’ll be right back.”

Even my wife, who has the patience of a saint (after all, she married me), was getting visibly agitated. “I’m glad you said something,” she said. “I was ready to get up and leave.”

That’s when I knew that it had gotten serious – if she was willing to risk being rude (she’s quite realistically the nicest, most patient person I’ve ever met), then she had been pushed way too far.

The sales guy came back with the sales manager. “I’m told that you have some questions,” the manager said.

Wait, what?!

Despite my gut instinct to walk out the door, I stuck it out. I explained everything – again – and confessed that I didn’t understand why this was such a difficult process. I mentioned – again – that I was here because of the Navy Federal buying program, but hadn’t yet seen anything to indicate any arrangement.

The sales manager left with the sales guy, and I looked at my wife. “What are you going to do?” she asked.

“I have a limit. If they don’t come back with a number that meets that limit, I’m out.” She nodded, and started to collect her things.

The sales guy came back with a single piece of scrap paper with a hand-written number on it. No financial terms, no fine print – hell no print on it at all except for the handwriting. it was $500 less than my maximum price. We shook on it.

Begging for 5s

This is where we began our story. As they say, though, it ain’t over until the fat lady sings.

While we were waiting for the “finance guy” to finish us up, the sales guy came over with some more paperwork. He showed me a sample satisfaction survey, and wanted to explain to me how it “worked.”

On the front of the survey, there was a big red “A+” with a circle around it. Seriously, like a grade on a homework assignment. On the back of the piece of paper was the exact same survey questions, but this time with a giant “F” circled. The only difference in the same sheet was that instead of “Excellent”, some of the circles had “Very Good” chosen.

The sales guy launched into a 20 minute lecture on how important it was that no matter what, we circle the “Excellent” in all categories. The implication was that he wasn’t going to get paid or, perhaps not quite so drastically, was going to get penalized in some fashion (no bonus? It wasn’t very clear).

There were open-ended comments fields, of course, and he explained how the survey was used. “Ford reads the comments,” he said, “so you can write anything you like in here. But these actual numbers – ” he tapped the 1 through 5 digits in the main questionnaire – “all have to be a 5. Anything less than a 5,” he turned over the sheet to show the “F”, “is a fail.”

“I hate to do this,” he said, “but I’m actually begging you to circle all 5s. If you want to say anything, please put it into the comments, but at least circle all 5s.”

He then asked me to sign the sample survey sheet, which indicated that I would, in fact, fill out a survey. I signed, thinking that I would go ahead and fill it out.

Upon Reflection…

Footage from the security camera at the dealership

Since that time, I have to confess that I’ve been getting extremely agitated about this whole “survey” thing. Ford has been sending me surveys to fill out, but every time I see the reminders I grow increasingly upset.

These customer surveys are supposed to mean something. When corporations place pressure like this on their employees, bad things can happen. Just look at what happened to Wells Fargo, for instance. It doesn’t take someone versed in research methods to know that there’s no room for “improvement” when you force your employees to beg for a perfect score!

As I thought about it, I began to get really angry. If I filled out that survey in this fashion, there would be no way for anyone to know that this kind of bait-and-switch is going on. I called Navy Federal’s car-buying program and explained the situation.

“They didn’t show you the truck invoice? Or Navy Federal’s negotiated price?” the agent asked, sounding incredulous.

“Nope.”

“I’m guessing you didn’t buy the truck?” she asked.

“No, I did – we came to an agreement on the value of the truck.”

“But you don’t know what the price should have been?”

“No.”

As I was speaking to her, I began to realize just how much the dealership had went far outside the boundaries of what was supposed to happen. I began to feel far more suckered than before.

I should be clear here – I found a truck that I like and I believe I paid a price that was a fair value for what I got.

I do think, however, that as I reflect upon the survey for customer service that there is no way that I can in good conscience give an “Excellent” rating on those questions. I do not believe that “Ford reads the comments” and therefore any issues that I may have had would be resolved regardless of the “all-5s” score.

Why?

Interpreting Surveys

Because that’s not how surveys get interpreted.

Well-formed surveys want to make sure that people are

  1. Answering correctly (i.e., they know what a ‘1’ means and they know what a ‘5’ means), and
  2. Answering truthfully

In order to do this, good surveys will make sure that questions in the survey are formulated so that there is consistency across the board. For instance, if you were to answer a question such as “I thought the sales person treated me well” with a “5 – strongly agree”, but then answer the later question “I thought the sales person did not respect me” with a “5 – strongly agree”, obviously something is wrong. These kinds of “validation checks” are important to know if the survey is valuable or not.

Likewise, if you answered that first question with a “5” and then wrote in the comments that you thought the sales person was trying to rip you off and was a liar (I wouldn’t write that in this case – it’s just an extreme example), there is a strong inconsistency here. A good evaluation of the survey would indicate that the “5” is less meaningful, because the respondent took the time to write out a comment, something that takes considerable more effort than circling a digit.

So, in a situation like this, what’s the point of circling the 5 in the first place?

The answer has to do with how the data is tabulated. Numerical values are stored separately from the open-ended questions so that they can be tabulated properly. Often this means that the comments are not actually associated with the numbers at all.

Okay, so that aligns with what the sales guy says, right? Yes, yes it does.

But here’s the thing – often times people do not read the comments in surveys such as this unless there is some numerical indication of a problem. “Why did we only get a 3?” “I don’t know, what do the comments say?”

While it may or may not be the case, there is a strong argument to be made that the numerical aspect of the surveys simply do not matter, but then neither do the comments since there is no way to associate these values with actual customer satisfaction.

If it doesn’t matter, and there can never be any changes made as a result to filling out the survey, why do it?

Sure, I signed the sample survey, because I said that I would, but the more I think about it, the more difficulty I have with going through with it. I am simply not capable of lying about the experience in order to fulfill some quota. I don’t think it will stop the dealer from doing the same thing over again, and I certainly don’t wish to reward behavior like this.

A Strongly Worded Letter

As is custom, I did receive a letter from the Internet Sales Director from the dealership. It was boilerplate, but it at least gave me an opportunity to provide a feedback experience of my own crafting:

Hello <Internet Sales Director>,

Thank you for getting in touch with me. I apologize for the delay in my response, for I was not sure exactly how I was going to write about my experience at Maxwell Ford.

First, let me say that I hope that <first sales guy> is feeling better. Unfortunately he had to leave the dealership ill without following up with me about an appointment, and <the salesguy I dealt with> was kind enough to juggle my needs along with a few other customers at the same time. The weekend after Thanksgiving is certainly a busy time and I believe he managed to help me sort out the test drives quite well. He was always polite and friendly.

Having said that, I would be remiss in not explaining my severe displeasure in working with Maxwell Ford’s car-buying experience on behalf of Navy Federal. In short, there was no difference between this “car-buying” experience and the normal torment that one goes through with a typical buying one. Despite wanting to talk about the program with both <the salesguy> and the sales manager (whose name escapes me), these requests were ignored. In fact, even though this was supposed to be a clean, cut-and-dried, “here is the invoice paperwork, here is the Navy Federal-agreed price,” no such conversation ever took place (again, despite repeated requests).

During the final stages of the buying process, I mentioned three times that I was going through Navy Federal, and three times <the salesguy> came back with Maxwell Ford financing instead. One of those times the price actually increased! This is not what I was led to understand from the experience, and I even expressed my irritation to <the salesguy> at the time, but to no avail. As far as I’m concerned, Maxwell Ford failed to provide an above-the-board interaction.

Navy Federal has followed up with me about my experience, and I have already expressed this to them. I have also been given numerous opportunities to provide Ford feedback, but <the salesguy> made it very clear that anything other than “all 5s” would result in “bad things happening.” A 20 minute lecture on how important it is to provide 100% stellar feedback “in order to improve” does not mean anything, of course. At this point, it appears the most positive thing I can do for <the salesguy> and Maxwell Ford is to not complete a survey.

Please do not misunderstand me – I have no anger or ill will towards <the salesguy>, nor am I unhappy with the truck. <the salesguy> was completely professional, polite, if not overly solicitous. I may have paid more for the truck than what the car-buying service would have negotiated, I may have paid less. I don’t know – which is exactly the point. I feel that the interaction, despite all the positives, wasn’t honest, and that the relationship with Navy Federal is a bait-and-switch to get people into the door and talk people out of the negotiated rate and into Maxwell’s financial offerings.

As of this writing, Maxwell Ford is 100% paid for the truck, and the only interaction that needs to happen now is that the title be properly sent to Navy Federal as the lien-holder, and that the Texas registration and plates get sorted. I will not, however, be returning to Maxwell Ford under any circumstances, unless there is some legal obligation that needs to be completed. The true shame of it is that I am an extremely loyal customer when treated correctly, and there are several thousands of dollars of improvements I have planned to make to the truck that would have otherwise gone to Maxwell Ford.

Otherwise, I hope that Maxwell Ford improves its internet sales procedures so that future customers will not have the same experience that I did. I certainly won’t be recommending the dealership to anyone. All the polite, solicitous salespeople in the world don’t matter, if they simply mask underhanded policies and procedures.

Very Truly Yours,

J Metz, Ph.D

True To My Word

I’ve redacted the names of the people involved at the dealership, but left the dealer’s name on purpose. I do not recommend them, I do not trust them, and I don’t think anyone should believe that they will have any other type of experience by going there. As a matter of fact, as I was writing up this blog article, a friend who lives in the area just told me that they are looking for a new car as well, and I discouraged them from going there.

As someone once told me, a recommendation to not go somewhere is often more valuable than a positive recommendation. I certainly believe that to be the case here.

As of this moment, there isn’t more to tell the story. If there are any updates or epilogues, I will be happy to share. But at the very least, I don’t think I’ll be filling out any surveys for Ford any time soon.

Comments

  1. Your first mistake was not understanding the nature of the relationship. Car sales are one-time relationships, not long-term relationships. The data the car dealers have is that if you don’t purchase the car on your visit, they will never see you again.

    Your other mistake was not understanding how dealerships make money. They make very little money on the actual sale. The biggest profit contribution comes from upsells (dealer installed accessories like trim, floormats, etc., coating package), extended warranties, and financing. They really *need* you to finance through them in order for them to make big money.

    The solution is to vote with your dollar bill and your feet. As soon as the dealer does not meet your expectations, you tell them “you have 1 chance to fix this or I walk”. Like all threats, you *must* be willing and able to follow-through.

    1. Author

      I understand the relationships perfectly well, actually. What I didn’t understand (or know, since I had not done it before), was how the nature of the car-buying process was supposed to work.

      The person who originally contacted me from the dealer was ill and wasn’t there when I arrived (as I mentioned in the post). The second guy who came up and helped us needed to be told what was going on, so that he could pick up where the other guy ‘left off’, so to speak. I didn’t understand when the paperwork situation was supposed to happen, and thought that what we were supposed to was find a vehicle, and then we would begin talking about the car buying documentation.

      That is, after all, the whole point of this – yes, the dealers make money through the financing, but the point of the service is that they get a nearly guaranteed sale (physical time on the lot costs the dealer money) with very little risk of non-payment, collections, etc. It’s a trade-off that they accept. And Navy Federal recommends “vetted” dealers for this very purpose.

      I should have left sooner, it’s true. When the guy came out the first time with the financial negotiations, I gave him the opportunity to come back. When he didn’t, I should have left, it’s true, and that is on me (agreed).

      But what kind of blog post would I have written? 😉

Leave a Comment