mkaresh

In late 2005 I started conducting my own reliability research. I'm reporting absolute stats like "times in the shop" that will make the differences between cars much clearer. Relative ratings obscure too much--how large is the difference between "better than average" and "worse than average"? I’ll also be updating results four times a year, so there will be information on new models sooner.

SUV owners tend to be more interested than most in this research. For example, I already have 20 Audi Q7s in the panel, more than any other 2007 Audi. I similarly have more GL450s in the panel than any other 2007 Mercedes, but still need a few more signed up before I start collecting data.

To encourage participation, panel members will receive full access to the results free of charge.

Details: Vehicle reliability research

Comments, questions, and suggestions welcome.

oknish

Will you allow me to share your research results on forums like these for benefit of others?

mkaresh

My current policy is that someone can post information on up to three models in a thread. I'm all for people using the results to help others, but don't want to see wholesale re-publication.

Overall I'm trying to find a way to balance helping others with giving people a reason to participate. Because only if people participate will there be results to help others with.

Make sense?

oknish

I agree and you do make sense. How about if we discuss only information for GL since that is what this board is dedicated too? That meets your requirement for not posting information for more than 3 models yet at the same time allows current and prospective GL owners to get some details and hopefully even participate in your research.

If that works for you then I will be signing up today.

mkaresh

I say three so people can discuss the main vehicle they're interested in and a couple of competitors, to provide context. So this sounds fine to me.

The only significant restrictions I might add in the future would pertain to additional stats that I'm not even providing yet, such that the major stats would be open for discussion but some of the details might be restricted to the site.

I'm sorting this out as I go. But what won't change is that the primary goal is to help people as much as possible. And it will only be possible to help people if these people help provide the data.

This might seem like so much fluff, but it's really not. Look at J.D. Power. Even the people who fill out their surveys have access to few of the results. Their focus is on serving the manufacturer, not car buyers directly.

boxboss

This will be difficult. Unless you allow free access to the data, how will people benefit from it? You do have a laudable goal, but how to disseminate it and protect yourself in the process? Advertising? Sponsorship? If it's auto related, you will be accused of bias, regardless.

I do like your approach of "times in the shop" rather than JD Powers "initial quality" or "buyer satisfaction." Good luck!

Abscam

w/o a good and varied sample set it will be biased data. For instance if you only grab data from board posters (auto freaks) you are already with a bias. Those w/ issues re more likely to respond than those w/o issues as well.

W/o a mass reach out to all vehicle owners via direct mail, I'm not sure how you could get a large sample set. Good luck.

mkaresh

Full access will be possible one of two ways, only the first of which is currently available:

1. Help provide the data as a member of the data

2. Pay for access

People who want the information, but who want people other than themselves to provide the data, will financially support the project. So those who do make it possible by helping out won't have to pay anything.

It's probably all a moot point with regard to Mercedes reliability information, anyway. Only one GL has been added to the panel today. Mercedes owners are tough, possibly because people who care about reliability information don't often buy Mercedes.

mkaresh

My methods greatly restrict bias. I just wrote about this earlier today in a blog entry:

http://www.truedelta.com/blog/

The sample sizes are coming along nicely. They'll get where they need to be.

oknish

I agree with what you are saying, Abscam, but I think he has to start somewhere. And, IMO, frequent board posters tend to have a more balanced view. But I could be wrong.

I will give it a try and see what happens.

mkaresh

On the sample size, as you say I have to start somewhere. Results, even with small samples, have increased the rate at which people join up. That said, I've been pleasantly surprised by the stability and face validity of the results given the currently small sample sizes.

On the bias issue, read the blog entry I posted a link to. Most importantly, I'm not conducting an opinion survey, and people can't pick and choose when to respond like they can with CR and JD Power.

I thought through and addressed many of these issues before getting started. Others have come up by paying careful attention to the data, and in discussions like this one. They keep me on my toes, and ensure that I don't deceive myself that everything is perfect.

oknish

mkarseh, what is your end goal with this research? Just curious.

mkaresh

Nothing too complicated, just far better vehicle information than anyone else is currently providing.

Before I launched the site I actually pitched the idea for the reliability research to J.D. Power. They didn't feel the need to do anything more than they were already doing. They're already getting every dime the manufacturers can afford to spend on reliability info, so additional research would just raise their cost without raising their income. I then called Consumer Reports, but they said everyone there was too busy to speak with me.

Just one of those cases where if you want something done you've got to do it yourself. The hardest part was teaching myself to program. I already had a solid background in survey research from my Ph.D. at the University of Chicago and associated work at the National Opinion Research Center.

Drives my wife crazy that I'm so focused on the product rather than income, but I figure if I get the product right then rest will take care of itself. Especially since I don't have the overhead of a CR or JD Power.

oknish

more power to you, brother. If you build it they will come.

mkaresh

That is the hope. I just wish they'd come sooner! Recently passed the 12,000 vehicle mark, but need many more.

With the GL450, up to ten. A few more and I'll start collecting data on it.

mkaresh

Still haven't quite gotten that "few more."

mkaresh

Closer, up to 13.

Latest results, though sadly still no Mercedes:

TrueDelta Vehicle Reliability Survey results

Newbie_2_MB

OK, you've got another GL signed up.

AsianML

W/O issues here and I joined up.

mkaresh

Thanks.

There are now 20 signed up, so I started collecting data. I'll need a few more participants to provide results, so the goal is results for the GL in November.

StevethePilot

Enrolled, first (I guess "and second") repair listed for the GL.

mkaresh

Thanks. The GL won't be in the August results, since I just started collecting data on it, but hopefully will be in the following set, in November.

This will depend on how many owners sign up and participate.

For the details, and to sign up to help out:

Vehicle reliability research

TrueDelta will have asterisked (for marginal sample size) results for the Audi Q7 next month.

mkaresh

Getting close to the November results. A few more participants could make all the difference.

mkaresh

The last round of data collection before the November results starts this week. 27 owners are signed up for the 2007 GL, so it's still iffy whether it'll be included in the results or not.

Stumblefoot

I just signed up this afternoon.