How do I create an algorithm to rate products?

June 26, 2008 5:47 PM

This will expose my complete ignorance of math, but hey. I need a simple algorithm to calculate and display overall individual grades for similar products.

Each product will be judged in four equally weighted categories -- let's call them A, B, C, and D. I'd like to assign a range within each category, and rate each product on it. I'd also like the final grade to be based on a 100 point scale. The goal is to be able to easily rate one product against another. For instance, if I was to use a 1-5 range for each category, it might go something like this:

Product #1 gets a 3 in A, a 5 in B, a 4 in C, and a 4 in D. (16 out of 20, or 80/100)

Product # 2 gets a 4 in A, a 5 in B, a 5 in C, and a 4 in D. (18 out of 20, or 90/100)

The problem with those values, however, is that they yield far too tidy final scores. (80, 90, 55, 65, etc.) I'd like more varied scores, but without fractional numbers. (No 82.5, for example.) The value range can be pretty much anything, but preferably not too large.

Any thoughts?

posted by william_boot to education (19 comments total)

If you have four categories, and a final 100 point scale, each category controls 25 points of the final score. The only numbers that will divide 25 evenly with no fractions are 5 (which you're using now) and 25.
posted by demiurge at 6:01 PM on June 26, 2008

The only thing I can think of right now is to rate each category out of 25.
posted by theichibun at 6:09 PM on June 26, 2008

Surely there's some math genius out there who can bend this field to my needs!

What about using an odd range -- say 1-7 -- and then a correcting multiplyer?

Ack...even my questions are nonsensical. Damn you, math!
posted by william_boot at 6:14 PM on June 26, 2008

demiurge has it. If you want the final score to be out of 100 with no fractions, and there are four categories, the only possible value ranges are 5 and 25. If you want something else, you have to drop one of those requirements.

Can you add a category? With five categories you have a few more choices for the value range: 20, 10, 5, 4, or 2. (Multiplying each category score by 1, 2, 4, 5, or 10, respectively.)
posted by equalpants at 6:24 PM on June 26, 2008

What if you had a non-linear range? If your four categories had potential values of 0, 1, 4, 9, 16 and 25 (say), then you could have a wide range of different scores.

You don't have to list the scores in each category as being 0, 1, 4, 9, 16 and 25---they could be from 0-5, but then before calculating the overall score, you convert to the new scale: 2 becomes 4, 3 becomes 9, et cetera (in this case you just square each number). So in your example, a product could still get a 3, a 5, a 5, and a 4, but its final score would be 9 + 25 + 25 + 16 = 75, while a product getting a 4, a 5, a 5 and a 4 would have 16 + 25 + 25 + 16 = 82. You don't actually get a larger range of possible scores, but the scores look more different.

Of course, once you've accepted non-linear scores, there's no reason it has to be 0-5: it could be 0-7, converting into 0, 1, 3, 6, 9, 14, 19, and 25 (numbers I just made up) for example. But if all you want is for the scores not to look so "tidy", you don't need to do this even.

How does that sound?
posted by goingonit at 7:02 PM on June 26, 2008

I'm not sure I understand exactly what you are looking for. You say "I'd like more varied scores, but without fractional numbers." Does that mean more variation within the final value out of 100 (the possibility of getting anything--or at least more unique values--between 1-100 as a final score)? Or does it mean more "untidy" numbers (59 vs. 60)?

I may be talking out of my ass here (c'mon, there's gotta be some real mathematicians on ask mefi, where are ya folks?), but I think that there is only a certain amount of variation you're going to get because your initial starting set of values is small. If you increase your rating scale for each of the A-D values to something larger, that will increase the variation out of the final 100. But you will always start to see the same values repeated over and over if you are dealing with a small set to start with. I believe it would be expressed by factorials, like 4! * 10! (all the possible variations with four different values each having a range of 1-10...), if my math serves me...or if I'm doing my math justice is more like it...

However, if you just want "untidy" numbers, you could do something like use the 1-7 range you suggested--if you mapped that into 100, you'd have to round off or just chop off the decimal, but you'd get non "tidy" numbers (this is very subjective, of course).

For example, if (A, B, C, D) each had a value within the range of 1-7, you'd get something like (2, 5, 1, 7), which you could either multiply by 14 and get something roughly approximating the division of 100/7, or you could use a more accurate value (14.2857...) and then round off after you averaged that out. So, with the above example, you might have something like (28.5714, 71.4285, 14.2857, 99.9999), and then averaging those you'd get 53.571375, which you could easily round up to 54 (i.e. 54/100).

Am I answering your question?
posted by dubitable at 7:07 PM on June 26, 2008

I'm also confused about what you want. If you have 4 equally-weighted categories scored 1-5, there are only 20 different possible final scores regardless of scale. You can get 100 possible final scores by scoring each category 1-25, but then you've committed the survey cardinal sin of thinking that human beings can consistently tell the difference between (say) 13/25 and 14/25.

The only other way to get more possible final scores is to weight the categories differently. Conveniently, if you apply weights in multiples of 5%, you don't even have to round/floor/ceiling the results. Let's say you decide ahead of time that category A is responsible for 15% of the final score, B is 10%, C is 35%, and D is 40%. Someone gives a score of (1,4,3,5) (for A,B,C,D). So that's:

(1/5) * 0.15 + (4/5) * 0.10 + (3/5) *0.35 + (5/5) * 0.40 = 0.72, final score of 72/100. I'm pretty sure you wouldn't have to round for any set of scores A,B,C,D, but I'd want an actual math geek to check that. (Quick hand-wavy reason: 20% * 5% = 1%, which is the smallest increment you can add to a score, therefore even percentages.)

If your weights aren't quite so tidy, you could end up with a more untidy final answer. For the sake of argument, your example, 82.5. The floor function just throws away everything after the decimal point, and you end up with 82. Floor functions have the advantage of not rounding up an imperfect score to 100.

But to your initial question? If everything has equal weight, then there are as many possible final answers as there were initial boxes to check (1-5 for A, 1-5 for B, etc). If you reported the results as x/20, you'd see the full range of whole numbers (well, 4-20). Translating to a 100-point scale just moves the 20 different scores so they're 5 points apart.
posted by range at 8:24 PM on June 26, 2008

Echoing range, you can assign a set of weights w_A through w_D to the categories, such that they sum to 1 and each weight is between 0 and 1 inclusive. You can arrive at a net score for an item via a linear combination of the corresponding weights and scores, s_A through s_D, for that item, i.e.
net score = w_A*s_A+w_B*s_B+w_C*s_C+w_D*s_D
and rounding as necessary. This approach has the benefit that different people can assign different weights to the score data in accordance with their personal relative importance of the categories.
posted by harmfulray at 8:47 PM on June 26, 2008

What about using an odd range -- say 1-7 -- and then a correcting multiplyer?

This won't change the fact that you want the same weight for each of four categories and you want the total score to sum up to 100. That means that each category has to translate to a final range of 0-25 before the scores are summed. if you had a range of 1-7, each point given to a category would translate to a (25/7) increase in the final score. (25/7) is not an integer.

If you forgo a final score total of 100, you can have any range you want for the individual categories and just add them up without any type of multiplier.
posted by demiurge at 9:10 PM on June 26, 2008

Also, I would caution you against using non-linear scores, like in the scheme goingonit presents. I think this would be very confusing even for people with some background in math. Product ratings are not usually put in those terms.
posted by demiurge at 9:14 PM on June 26, 2008

How does that sound?

It sounds great, if I understood it, goingonit. Would such an approach be essentially the same as weighting each category differently? Or is it just a more clever way of getting to 100?

And range and harmfulray, I have considered weighted categories, but I prefer not to do so if at all possible. Basically, these products are best graded in four categories, and each category is equally important. So weighting is a bit of a cop-out, and I'd like to avoid doing so if at all possible.

As I mentioned, I can score each category one any point scale I desire -- a 5 pt, or 6 pt or 7 pt, etc. But I need to keep the number of categories to four, and I'd like to keep the overall score range as 0-100. I guess I was hoping that there would be an easy algorithm which, for instance, spelled out exactly what each value point was worth, then gave me a handy percentage or fraction to multiply the sum against to get my tidy 100 point scale. (Maybe that's what goingonit is getting at.)

Ah, it all sounded so simple in the abstract. Again, damn you, math!
posted by william_boot at 9:29 PM on June 26, 2008

Open review, maybe range has the most sensible option, with his weighted option option:

The only other way to get more possible final scores is to weight the categories differently. Conveniently, if you apply weights in multiples of 5%, you don't even have to round/floor/ceiling the results. Let's say you decide ahead of time that category A is responsible for 15% of the final score, B is 10%, C is 35%, and D is 40%. Someone gives a score of (1,4,3,5) (for A,B,C,D). So that's:

(1/5) * 0.15 + (4/5) * 0.10 + (3/5) *0.35 + (5/5) * 0.40 = 0.72, final score of 72/100. I'm pretty sure you wouldn't have to round for any set of scores A,B,C,D, but I'd want an actual math geek to check that. (Quick hand-wavy reason: 20% * 5% = 1%, which is the smallest increment you can add to a score, therefore even percentages.)

Can you math wizards weigh in on that approach?
posted by william_boot at 9:35 PM on June 26, 2008

Try A=25, B=19, C=13, D=7. That way you'll have equal weighting, linearity, a top score of 100, but varied-looking scores below that (70, 76, 82, 88, 94, etc).

If you allow slight nonlinearity, then A=25, B=17, C=10, D=4 will give even more variation (100, 92, 85, 84, 78, 77, 76, 71, 70, 69, 68, etc).

A small variation in weighting between the dimensions will give more still: e.g. 25/20/15/10 for one dimension, 25/19/13/7 for another, 25/18/11/4 for another, 25/17/9/1 for another. Then possible scores will include 100, 95, 94, 93, 92, 90, every score between 32 and 90, then 30, 29, 28, 27, 26, 22.

If you want the possibility of a zero score, then 25/18/11/0, 25/17/9/0, 25/16/7/0, 25/15/5/0 would also cover a wide range.
posted by rationalist at 9:55 PM on June 26, 2008

Ah, I see you have five scores for each category. Given that, adding E=1 for the first method above, and E=0 for the next two will add even more variation. (N.B. I'm using A-E for the scores, not for the categories -- sorry for the confusion.)
posted by rationalist at 10:10 PM on June 26, 2008

I would caution you against weighting categories differently just because you want 'untidy' final scores. If this is for anything remotely serious where people will use the numbers for decisions, using weighting for this purpose is unprofessional. If this is simply for an example or demo, then weighting categories differently like range described would work.

rationalist's idea is workable, it moves the minimum score up so you can use a different divisor. You'd have to note that the minimum score isn't zero, since most people expect that to be so.
posted by demiurge at 10:27 PM on June 26, 2008

william_boot:

I guess I was hoping that there would be an easy algorithm which, for instance, spelled out exactly what each value point was worth, then gave me a handy percentage or fraction to multiply the sum against to get my tidy 100 point scale. (Maybe that's what goingonit is getting at.)

I still don't get it. Are you looking for something that provides you with the ability to have a large range of values within 1-100 (or 0-100 as the case may be), or values that look "untidy" (which, again, is really subjective)? It seems like rationalist's scheme gives you a lot of possibilities but I agree with demiurge that it's sorta confusing and unhelpful if humans on the other end have to comprehend how it works.

If you want something that maps tidily to a 100 point scale (without getting decimals), then that must be something that 100 is easily divisible by (2, 5, 10...). But it seems like you are unhappy with getting values that are "neat," right?

Can you please restate your idea, what you are looking for, as clearly as possible? And maybe some more context?
posted by dubitable at 11:20 PM on June 26, 2008

The only way to get more varied scores within your parameters is to grade each category out of 25.
posted by sid at 6:26 AM on June 27, 2008

In my opinion, discard the notion that untidy scores are desirable. They really are not, people like seeing things in rounded numbers. If you have multiple people rating a product, their average score will come out pretty "untidy", so don't worry too much about it.

However, please do allow people to vote on a scale from 0-5. That will allow "more variety" as the tallied score allows for a true zero, rather than 20% (all 1s) being the lowest possible rating.
posted by explosion at 7:36 AM on June 27, 2008

Yeah, I should have clarified: You really only ought to weight the categories differently if you actually feel they carry different weights toward the final score. Note that scoring category A from 1-7, category B 1-5, category C 1-26, etc. is also applying different weights to them, if you just add up the numbers without renormalizing (in which case, there's no point in having the different scales in the first place).

Given your responses in-thread, I'm starting to come firmly down on the side that says your original question doesn't make any sense and is numerically impossible. If you score 4 things from 1-5, there are only 20 possible final scores, the end*. If you want to see all the whole numbers represented, and not jump by 5's, then report them on a 20-point scale. Weighting things, or really employing any other scheme, just because you want to see more granularity on your 100-point scale is capricious and mis-represents both the precision and the quality of the reporting that your judges are going to be actually doing.

*For pedants: only 17 possible scores, since you can't score a zero in any category and therefore scores 0-3 aren't attainable.
posted by range at 2:04 PM on June 27, 2008