• 0 Posts
  • 19 Comments
Joined 1 year ago
cake
Cake day: June 12th, 2023

help-circle


  • I just want to jump in here as the whole thing about the tonnes of factual errors stuff…

    A lot of the allegations about the accuracy of their data basically came down to arguments about the validity of statistics garnered from testing methodology; and how Labs guy claimed their methods were super good, vs other content creators claiming their methods were better.

    My opinion is that all of these benchmarking content creators who base their content on rigorous “testing” are full of their own hot air.

    None of them are doing sampling and testing in volume enough to be able to point to any given number and say that it is the metric for a given model of hardware. So the value reduces to this particular device performed better or worse than these other devices at this point in time doing a comparable test on our specific hardware, with our specific software installation, using the electricity supply we have at the ambient temperatures we tested at.

    Its marginally useful for a product buying general comparison - in my opinion to only a limited degree; because they just aren’t testing in enough volume to get past the lottery of tolerances this gear is released under. Anyone claiming that its the performance number to expect is just full of it. Benchmarking presents like it has scientific objectivity but there are way too many variables between any given test run that none of these folks isolate before putting their videos up.

    Should LTT have been better at not putting up numbers they could have known were wrong? Sure! Should they have corrected sooner & clearer when they knew they were wrong? Absolutely! Does anybody have a perfect testing methodology that produces reliable metrics - ahhhh, im not so sure. Was it a really bitchy beat up at the time from someone with an axe to grind? In my opinion, hell yes.








  • LMG is 200 odd people. You could say Nvidia with >10k employees is colossal perhaps, or you could say Cisco with >80k is. 200 people is barely a medium sized enterprise. Christ, the company I work for is nearer 2k people.

    And in any of those cases, the size of the entity doesn’t matter - its a very common journalistic practise to seek comment. I tend to think that because GN chose not to, that it veered away from a purely journalistic exercise because they are essentially a competitor. They still have the right to publish it and make the claims they want but it changes the framing of the video quite a bit for me.

    Bear in mind that I think all benchmarking is over hyped in its reliability and importance so pretty much none of that side of things holds any real credibility with me at all - so i’m mostly judging GN based on they way they’ve presented this issue.

    Finally, Linus didn’t sell the startup’s prototype, LMG did. And unless you’re ignoring the whole trail of fact on that its pretty clear it was a stupid mistake but also how that came to be isn’t some evil genius plan.






  • I would contend that most of the YouTube tech channels, even the ones with great reputations for quality such as Hardware Unboxed and Gamers Nexus are barely educational as well.

    While they can point to fewer mistakes made as a result of their methodologies, I don’t believe there is any real scientific value in the conclusions they reach from testing that is far too limited with far too small a sample size. They can paint broad recommendations - product A should be better than product B because our testing showed 20% better numbers.

    But when the metric variances between products are small, none of the testing methodologies can really tell you which of the products will work better in your system. They haven’t tested enough of them, and in enough situations to have a clue. And I think any of them claiming that there is inherent value to their methods are really just defending their product which is the video they’re getting eyeballs in front of so they can make money from advertising.





  • I guess its not just anonymous surveys though is it, its that, plus a low turnover, plus 1-1 feedback sessions and other things I’m probably forgetting. But don’t overlook the conclusion where if you read between the lines, more personnel changes are likely as a result of this.

    I think your last point is highly unlikely, there is too much light on this for the issues alleged by Madison to be true, and for LMG to do nothing. But the tricky thing is that the full truth about Madison’s issues is unlikely to be made public, and equally the full action taken by LMG will not be made public. HR, and employment issues kinda require privacy - this isn’t a government or public department, its a private company with responsibilities to employees past & present mandated by law.