If geeks love it, we’re on it

Battery life ratings: Time for a do-over

Battery life ratings: Time for a do-over

What? It's only been an hour and a half!

What? It's only been an hour and a half!

Quick! What do laptops and the Chevy Volt have in common? If you answered “bogus battery claims,” you got it. Batteries are more capable than ever, but their merits are being wildly oversold. Nine hours of battery life? 230MPG? Who authorizes this crap? Marketing departments have made battery life a race to the top of Bullshit Mountain, and the unaware consumer is paying for it with disappointment. We think it’s time that consumers are offered more reasonable expectations, and we think we have a solution that will do just that.

A growing chorus

The issue of bizzaro-land battery marketing was brought to the fore when AMD’s SVP and CMO Nigel Dessau called shenanigans in March.

“Most PC battery time metrics are achieved by looking at how long the battery lasted running a benchmark called MobileMark® 2007 (MMO7). This is a rating of battery life when your PC is running on average less than 5% utilized – or fundamentally idle,” he wrote. “Most PC makers don’t even turn Wi-Fi on for this test. Is this realistic based on how you use your PC?”

“If I want to know how long my battery is going to last, I want to know how long it’s going to last with me using it, not with it idle or doing nothing.”

Icrontic’s own Cliff Forster echoed Dessau’s sentiments when he wondered why consumers are willing to put up with the bogus numbers shoveled onto the market.

“Why is this happening? The truth will baffle you: OEMs test laptop battery life with critical components shut down. That means no WiFi and no advanced graphics,” he wrote. “To put it simply, today’s battery life claims are based on a bogus low power user profile. How do we allow this to go unchecked?”

Icrontic was certainly not alone in its opinion; the Wall Street Journal, the New York Times, Computer World and even Lenovo acknowledge Dessau’s observations as a legitimate problem.

“We don’t really like the fact that something is supposed to get four hours and users routinely say, ‘We divide that number by two and that’s what we get,’” said Lenovo segment marketing manager David Critchley.

Gathering some data

We wanted to go a step beyond punditry and tackle the issue for ourselves. Could we develop a battery life rating that more accurately depicted the consumer’s experience? After discussion, we thought it best to create three separate ratings for gaming, multimedia, and productivity.

To see if we were on the right track, we hit the Internet with a poll (now closed) that asked a simple question: What kind of laptop user are you? We let our readers, Twitter users, and corporate folk tell us what kind of user they are, and the answer surprised us.

what_kind_laptop_user

Next, we asked respondents to rate the importance of eleven distinct tasks on a scale of 1-5, with one being the least important. We made sure to cover a wide spectrum: Gaming, music, movies, browsing, blogging, pictures and writing are just some of the tasks we asked users to tell us about. We asked these questions to assure that our user’s usage habits reflected the categories they selected: Did productivity tasks outweigh multimedia tasks? Was gaming really that unimportant? This is what we found:

Browsing the web crushed all other tasks for the most important.

Browsing the web crushed all other tasks for the most important.

Flash gaming? Not so much...

Flash gaming? Not so much...

Document creation is 4-5 important for nearly 70% of those polled.

Document creation is 4-5 important for nearly 70% of those polled.

3D gaming was the single least important task amongst respondents.

3D gaming was the single least important task amongst respondents.

Watching media stored on discs came out a wash.

Watching media stored on discs came out a wash.

Movie sources beyond the disc proved much more important.

Movie sources beyond the disc proved much more important.

The trend for CD music leans towards "unimportant."

The trend for CD music leans towards "unimportant."

...Especially when compared to digital music.

...Especially when compared to digital music.

Image editing trended towards an important task.

Image editing trended towards an important task.

Over 50% of those polled thought file sharing was at least moderately important.

File sharing was pegged with moderate importance.

Social sites like Facebook and Twitter clearly trended towards "important."

Social sites like Facebook and Twitter clearly trended towards "important."

Interpreting the data

Now that we’ve presented the numbers as they are, we wanted to draw a few conclusions about what we found and highlight a few interesting results in the data:

  • Internet browsing is far and away more important than any other task. With 245 votes of 5, or “most important,” it is more than twice as important to users as document creation which received the next highest share of fives at 113 votes.
  • Of the five tasks which were rated as “most important” by those polled, Internet browsing and document creation (productivity) have more votes than the remaining three combined.
  • Even if you slant the numbers by adding all four multimedia options (CD music, digital music, DVD/Blu-ray movies and digital movies) together, the productivity category still receives more “most important” votes.
  • Gaming is a complete bust on the notebook. It received more votes for 1, or “not at all important,” than it did for “very important” or “most important” at 4 or 5.
  • Interesting: Digital music (iTunes/other) was deemed “most important” by users at a rate of more than 2:1 over music stored on a CD.
  • Interesting: We didn’t figure people would be doing much file sharing on their notebook, but nearly half of those polled gave it a 4 or a 5.

Most importantly, the specific usage habits offered by users in the importance polls reinforces the results to the “What type of laptop user are you?” question. Users clearly prefer productivity to multimedia, and multimedia to gaming. This is what we expected going into our analysis, and we’re pleased to see strong indicators that endorse the point.

Reacting to the data

Having sufficient cause to believe that notebook users fit into one of three archetypes, our next goal was to create a testing methodology that convincingly replicates their usage habits. In addition to a few benchmarks we feel are qualified to do that, we also have a few ideas about how the test results could be marketed to consumers.

The gamer

No matter which way you slice it, 3D gaming on a notebook was seriously unimportant to our respondents. In our data, it was 1.5-8 times more likely to receive a “one” than any other task we inquired about. Even so, the 22.8% of our respondents that awarded it a 4 or a 5 need to be considered, and gaming notebooks alone make the case for battery tests to accommodate.

Simple battery mechanics tell us that a battery’s life is a function of load, which means how we get to that point is largely immaterial. Accepting that, we propose the following:

  • Screen brightness 90%.
  • WiFi enabled.
  • Bluetooth enabled.
  • Loop 3DMark Vantage (3DMark 2006 for Windows XP Netbooks) on default settings until the battery expires.

The rationale for these test choices is as follows:

Firstly, an informal poll conducted by Neowin reveals that 62% of 1163 respondents run their display at 80-100% brightness. We have decided that splitting the difference is the most appropriate course of action.

Secondly, the importance people placed on browsing the Internet makes an active WiFi radio an indispensable part of the laptop experience.

Thirdly, we have previously identified that the current method of expressing a battery’s single-charge run time as an “up to X minutes” statement is inappropriate. We feel that a more honest testing ecosystem would make it into a “more than X minutes” statement. That is to say, testing should reflect a worst-case scenario. While Bluetooth is by no means prevalent, we feel it is appropriate to leave it enabled given this philosophy.

Lastly, we have chosen 3DMark because it creates a repeatable, consistent GPU load across a wide variety of GPUs and operating systems. Given that all new systems are shipping with Windows Vista and will soon come with Windows 7, a DirectX 10.x test seems most appropriate when considering the “more than” battery life philosophy. We make a lone exception for the Netbook with Windows XP, which cannot run DirectX 10 code; in these cases, 3DMark 2006 is the right choice.

The multimedia user

We have established that the multimedia user likes movies, music and pictures, so it would make sense to select a test that includes these tasks. To that end, we believe that the “Memories,” “TV and Movies” and “Music” suites from PCMark Vantage provide a consistent and repeatable platform that focuses on these activities. These three suites should be looped until the battery expires.

We continue to maintain that an active WiFi radio, an active Bluetooth radio, and a screen brightness of 90% are fundamental to breathing honesty into battery marketing.

The drawback to PCMark Vantage is that it is not compatible with systems that run Windows XP, as the Netbook may for quite some time. We are interested in PCMark 2005′s text, picture, video and 2D tests, but we are open to alternatives that may offer a superior routine.

The productivity user

We have established that the productivity user likes writing, reading and browsing. Once again we believe that PCMark Vantage can provide the appropriate tests with the “Productivity” and “Communications” suite.

These two PCMark suites specifically focus on networking, document creation and browsing, all of which we know to be fundamental tasks to the productivity user. The tests should be looped until the battery expires.

We continue to maintain that an active WiFi radio, an active Bluetooth radio, and a screen brightness of 90% are fundamental to breathing honesty into battery marketing.

Lastly, we are once again open to suggestions on an alternative that will appropriately test systems configured with Windows XP.

Selling the tests to users

We are not graphic designers.

We are not graphic designers.

If anything, Apple’s “it just works” mantra has proven that consumers don’t care about the “how” so much as they care about the “what.” Users clearly care about what they’re getting, but they aren’t interested in the minutiae of getting from A to B. This means that developing a new testing methodology means diddly if marketing doesn’t work to create more realistic expectations.

To that end, we propose a sticker that not only offers the battery performance for each type of user, but also offers a “combined” rating for users who may engage in a mixture of activities.

In the example logo to the right, we made the combined rating a mean of the three sub-scores, but the median between gaming and multimedia may be an appropriate choice as well.

Going forward

Many amongst Icrontic staff are equipped with the MSI Wind U100 or U120 units for trade shows, and I can guarantee you that none of us have ever–even with SpeedStep and 10% brightness–come within a stone’s throw of the six hours scribbled on the box. If users with handcrafted battery profiles and modded BIOS roms cannot get the advertised performance, something is seriously rotten in Denmark.

That stink has overstayed its welcome. Consumers have been bamboozled and dazzled by increasingly outrageous battery claims for years, and it’s time to make a change. We’ve proven that it’s possible, and now we’re calling on OEMs to make it happen: Use realistic tests, leash your marketing teams, and let’s work together for a more realistic battery rating.

Correction: The original run of this article indicated that one respondent, or .003% of those polled, reported that they did not fit in any of the offered categories. The correct number is 0.35%, but this reference has been removed accuracy and clarity. Please also note that our study has a margin of error of approximately 5.83%. We apologize for the confusion.

Comments

  1. BuddyJ
    BuddyJ I love the suggested icon. It may not be perfect but it certainly is better than what we have now.
  2. ardichoke
    ardichoke Great article Thrax. I'd love to see the industry adopt a system like that. Sadly I doubt they will unless some serious pressure is applied to them (in the form of lost sales for marketing inaccuracies). Maybe someone should set up a database in which people can post real battery life as determined by metrics like this. That was people can more accurately call manufacturers on their BS.

    EDIT: batteryrating.com is available :-D
  3. wthww
    wthww I wish there was such a system also. Lenovo claims nine hours of battery life for my ThinkPad X200 w/ the largest capacity battery available. running in powersave mode with BT & wifi on and active and brightness at fifty percent. I get closer to six hours. If I shut them off, about seven with the brightness at minimum. Only when completely idle and in powersave mode does it estimate nine hours or above. I'm quite happy with the machine anyways, considering its speed/battery life/weight ration is excellent.
  4. Cliff_Forster
    Cliff_Forster Robert,

    Grade A stuff. Consumer apathy has a role to play in all of this. I'm pleased to say that Icrontic is a tech consumer community that has taken a very public stance against this kind of blatant misinformation guided by the corporations we serve. They owe us better, and when enough people demand it, it will happen.
  5. Butters
    Butters Yeah Lenovo battery ratings are horrible. You have a compact laptop, I have an older generation T61P 15.4. Battery life is basically useless. Though, I'm not gaming, I'm lucky if I get an hour working on moderately cpu-intensive business apps with a 9-cell brand new. I rarely use it off its dock since its mostly used for work. MFG est 5.9 hours / 2 = 2.95 as far as I'm concerned is still a bit of a stretch.
  6. j
    j as I was told with my cell phone, charge your laptop more often and shut up.
  7. Gargoyle
    Gargoyle I can't wait until our laptops run on fuel cells and their energy usage will be rated in BTUs.
  8. ardichoke
    ardichoke
    j wrote:
    as I was told with my cell phone, charge your laptop more often and shut up.

    Hmm... cell phone runs for the entire day without charging... laptop runs for 2 hours if you're lucky. Somehow I don't see the correlation. Furthermore, you were BAWWWing about one specific cell phone (ignoring the fact that ALL smart phones need to be charged daily at least) NOT about battery life ratings in general. You, sir, are a dining room table and your argument is invalid.
  9. j
    j Ok well from a person who has to give a spec on battery life for my products I can say it's very difficult to place a single life of operation. You have to have a baseline and a standard to to test the baseline. Just because the baseline doesn't meet a few peoples opinion of how it should be tested doesn't make is wrong or inaccurate. But I'm all for challenging the status quo. So, it will be interesting for sure

    A word to the testing going forward. Make sure the testers us "NEW" batteries. If they're a year old tests results are invalid. The lithium in the Li-ion battery starts to degrade as soon as the battery is assembled. This is why they only last 2 to 3 years. Also, every charge further degrade the Lithium the battery.

    Future batteries (such as the ones in the Volt) Use a nano-phosphate Li-ion. Much better life span and higher current capacity. I would expect the next 5 years will show a revolution in battery technologies.

    Does any one know what power there computer draws. I can say on my desk top I got a Max of 125 watts even when playing TF2. I've been monitoring my PC for a month. What do laptops draw? I'd be interested in that.
  10. Gargoyle
    Gargoyle All I know is my cheap off-brand 65W adapter for my laptop gets hot enough that I worry about it starting fires. I'm thinking the laptop is pulling close to what it can offer.
  11. drasnor
    drasnor Computer manufacturers could always take a page from the aerospace industry:
    All of the aircraft and spacecraft designs I've had the chance to work on or get told about have electrical systems that are designed around these things called "load profiles." Basically, for each phase of operation and figure out which systems are running for that flight phase and total up the power draw. You get things like Engine Start: XXXX Watts; Cruise: YYYY Watts, Landing Approach: ZZZZ Watts. These numbers are useful for the designer because they determine minimum load capacities and duty cycles etc.

    If I'm buying a laptop, I think it would be neat to know:
    * The Watt-Hour rating of the battery.
    * The battery capacity degradation rate per year.
    * The power draw for various common usage scenarios: boot up, draconian low-power mode, radios on idling, radios on web browsing (20% CPU random disk accesses,) backing up hard disk (heavy disk access,) etc.
    With this information only a little simple math is required to figure out what kind of battery life you could get with YOUR usage patterns. It's not too complicated: if you're smart enough to care then you're smart enough to operate a 4-function calculator.

    -drasnor :fold:
  12. drasnor
    drasnor
    j wrote:
    Does any one know what power there computer draws. I can say on my desk top I got a Max of 125 watts even when playing TF2. I've been monitoring my PC for a month. What do laptops draw? I'd be interested in that.
    My 4-year-old IBM ThinkPad T42P draws 16-17W while on battery (minimum frequency scaling, mid-level backlight, BT and 802.11 radios set to min power), 25 W while on A/C in On-Demand, idling, and 35W at full tilt boogie.

    -drasnor :fold:
  13. UPSLynx
    UPSLynx Great piece Thrax.

    Consumer BS is so well entrenched, I doubt we'll see a truthful solution to this any time soon. Think of spec sheets for HD televisions. Contrast ratios are constantly smugged (dynamic contrast, anyone? Bullcrap.) as well as response rates.

    Spec is smudged so often, it's criminal.
    drasnor wrote:
    * The battery capacity degradation rate per year.

    This would be great to know as well, since batteries cost a frigging fortune once they crap out.
  14. ardichoke
    ardichoke Hmm... according to powertop (yay Linux) my laptop pulls between 17 and 21W when running on battery. I could probably tweak some more laptop-mode settings and get that down some more. I really wish MSI would release a larger battery for the EX630, the ONLY thing I have a problem with on this lappy is my battery life :(
  15. Daniel
    Daniel I just bought an HDTV with a 2,000,000:1 (2 milion) contrast ratio.
    If it even gets to 1:1000 on average I'd be shocked...
  16. Cliff_Forster
    Cliff_Forster Daniel,

    I know what you mean, deception in terms of actual contrast ratio ratings is another common practice. Audio amplification is another place where electronics manufactures like to occasionally play a trick on everyone by varying the way they measure a watt vs. a more stringent and realistic FCC suggested guideline. In the buisness I'm in we rely on an independent verification process to validate our capacity claims. One has to wonder why this is not required for more products?

    It all starts with an educated and motivated consumer.
  17. SoulEvolution
    SoulEvolution i cant wait for the heat-powered laptop
    wouldnt that be great? starts up by battery, purposely overheats due to tight componentry, ad uses that heat to power the battery! i think im on to something! has this been thought of before? 0_0
  18. UPSLynx
    UPSLynx
    i think im on to something! has this been thought of before? 0_0

    Only if you can keep the overheat from destroying the hardware.... or your house ;)

    The Chevy Volt. What a load of crap. 'You don't drive for more than 40 miles a day, right? The Volt can go FOREVER!!!1'

    Because the electricity it takes to charge it for those 40 miles is completely free.
  19. mirage
    mirage Excellent article!!!

    I think 3D gaming can be excluded from the battery life measurement with a footnote that it is required to have the laptop plugged in. My graphics card (8600M-GT) does not even switch to highest frequency on battery anyway.
  20. Thrax
    Thrax
    UPSLynx wrote:
    Only if you can keep the overheat from destroying the hardware.... or your house ;)

    The Chevy Volt. What a load of crap. 'You don't drive for more than 40 miles a day, right? The Volt can go FOREVER!!!1'

    Because the electricity it takes to charge it for those 40 miles is completely free.

    INFINITE MPG!!!!!111omg
  21. Gargoyle
    Gargoyle
    UPSLynx wrote:
    Because the electricity it takes to charge it for those 40 miles is completely free.

    Yeah! I mean, it costs about $0.80! Those assholes.
  22. Snarkasm
    Snarkasm
    j wrote:
    A word to the testing going forward. Make sure the testers us "NEW" batteries. If they're a year old tests results are invalid. The lithium in the Li-ion battery starts to degrade as soon as the battery is assembled. This is why they only last 2 to 3 years. Also, every charge further degrade the Lithium the battery.

    As it happens, that's an excellent argument to NOT use brand new batteries when testing. Using a new battery to run the expected power life profile for the laptop is just as idealized a scenario as is running the tests with all wireless radios disabled, screen at 10% brightness, and an idle processor.

    Run it with a 4- to 12-month old battery and give people what they can really expect over the early useful life of the laptop, not what they might get if they're lucky right out of the box.

    What a goofy suggestion.
  23. Gargoyle
    Gargoyle That means used batteries for different laptops would have to be used in the same specific lab conditions over that 4-12 month period in order to make the numbers comparable. Furthermore, since we're talking about the battery life of a laptop (and not just the battery), it would make sense to test the battery while it's actually in the laptop in question. For 4-12 months. Before it's on sale.

    No, I think it's much more practical to test new batteries to get comparable results, and inform users how to take care of their batteries to minimize wear out. I don't feel like that information is being advocated enough by manufacturers right now, because they'd rather sell you a replacement battery than make your current one last.
  24. j
    j Dras, 35 watts at full capacity. This interesting to me I thought it would be much higher. But I guess this makes sense since I think a laptop battery is 60 to 65maH. So that's 2 hours of use on max tits.

    Is there a power monitoring program I can install?
  25. j
    j Snark, you can't be wrong enough. The reason you use new batteries is so you know what the capacity is to begin with.

    I see your point, but if you start from an unknown capacity it's very hard to come to a reasonable conclusion. Perhaps after a few months the test should be run on the same batteries to see the degregation, but to do that you would have to log how many time you charged the battery.
  26. Snarkasm
    Snarkasm Just because a battery isn't brand new doesn't mean you can't possibly know its capacity.
  27. GnomeQueen
    GnomeQueen Great article, Rob! I know that I was super disappointed in the life of my laptop a few years ago- Dell told me that it would last for three hours, and I was lucky if I got an hour and a half. (It was also one of those batteries that might explode and set you on fire!). For a student, knowing your battery life is really important, because especially as finals time ticks down, you're going to want to study in places like libraries where there might not be tons of access to outlets. Listen up, battery companies!
  28. j
    j
    Snarkasm wrote:
    Just because a battery isn't brand new doesn't mean you can't possibly know its capacity.

    Your wrong. Every time a battery is charged it losses capacity. Battery Manufactures have a graph on this very thing and that's why they place a charge cycle count on it. Also, in the case of Li-ion Batteries if they are left on the self there capacity degrades, not by much but enough to screw up a test. The battery manufacture also has data on this.

    If the intent is to change industry labeling, then you had better had accurate test data. They will ask the very same things I've been talking about.

    I think Drasnor is right, Try to get them to place different hours of operation for different loads. This might be an easier sell. Then at least marketing is happy because they can play there game and put "9 hours" in big font on the advertising, but on the back it shows a performance curve based on common real world loading situations.
  29. Norge
    Norge
    j wrote:
    I think Drasnor is right, Try to get them to place different hours of operation for different loads. This might be an easier sell. Then at least marketing is happy because they can play there game and put "9 hours" in big font on the advertising, but on the back it shows a performance curve based on common real world loading situations.
    I think the point of this article was to avoid that type of marketing BS. The goal is to get rid of the "9 hours" in big letters on the front and replace it with something meaningful.

    I think the testing method the article chose seems like a pretty reasonable way to do things. Get new batteries of the same age, get a standard testing software, and present the results to customers in an easy to read format. Sounds like a good idea.

    Norge
  30. drasnor
    drasnor The power management tools that ship on IBM/Lenovo Thinkpads have a panel called "Battery Information". On that panel is the date of battery manufacture, FRU part number, serial number, design capacity, current capacity, wear level % (100 * current capacity / design capacity), and cell manufacturer.

    There was an unmaintained Windows XP tool called MobileMon that gave the same information for any ACPI-compliant battery, but these days I use Ubuntu and it's on the power manager battery information page.

    -drasnor :fold:
  31. Lee
    Lee I have been a longtime apple user and I have found their ratings to be accurate, if not less than what I can get during productivity use. I currently use a new Apple MacBook Pro 17" with a published 7 hour battery life and I have seen 9-10 continuous hours from it on a number of occasions.
  32. Bob
    Bob 1/283 = 0.35%, not 0.003%
  33. primesuspect
    primesuspect Thanks for pointing out the error.
  34. CANIBLOG2
    CANIBLOG2 I came in here to see this excerpt from your post myself:

    "Of the 283 respondents, just one (0.003%) felt that they could not be classified by the categories we had selected. We had expected the margin of error to be much higher than that, so a margin of just 0.003% let us know that we were definitely onto something big."

    But I guess you got to the reddit comment page first and already edited the original without indicating it. You should probably take a couple courses on statistics and journalism.
  35. Snarkasm
    Snarkasm Yes, because making one typo - .003 vs .003% - is definitely worth redoing all schooling.
  36. Uncle B
    Uncle B Depleted Uranium super batteries! Exposed just once on the net by a super patriot! Do they really exist? Can they possibly be as good as all that? Why can't the Navy Labs that produced them come clean, in hard times adnlet the common folk have them? Would they really solve the "Electric Car " dilemma? Even at the high weights given, the power outputs are astronomical! We need help here!
  37. Factopo
    Factopo Yeah, I saw that too.
  38. drasnor
    drasnor Stand back, I'm going to use Science. And if your super-battery relies on the radioactive properties of DU, you can just keep on hoping. Nobody is going to sell you radioactive materials hot enough to power anything.

    -drasnor :fold:
  39. Azati
    Azati I don't know about using a full on 3d benchmarker during a battery test. No one expects to play much more than solitaire on battery power.
  40. Snarkasm
    Snarkasm Except for professionals who expect to be able to do their work on battery power.
    Except for travelers who expect to be able to listen to music or watch movies on battery power.
    Except for everybody who wants to check email on the go or do some work down at the coffee shop on battery power.

    Cool generalization, bro.
  41. ardichoke
    ardichoke I think he was just referring to gaming on battery power Snarky.

    Cool ragetrip bro.
  42. Snarkasm
    Snarkasm Who was raging?

    Cool assumption, bro.
  43. Linc
    Linc
    Snarkasm wrote:
    Cool generalization, bro.
    I believe you misread his statement.
    Azati wrote:
    No one expects to play much more than solitaire on battery power.
    None of your counterexamples used 3D rendering, unless the professional was an animator or game dev.
  44. Snarkasm
    Snarkasm I recognize the misinterpretation - "play much more" was interpreted as "do much more than play."

Howdy, Stranger!

You found the friendliest gaming & tech geeks around. Say hello!