I wish there was an actual thriving business model like this -- just fixing most annoying bugs, for a price, of commonly used desktop software. Why proprietary software companies cannot or do not want to provide this service is over me. Perhaps I'm too much used to consulting.
Given that “fixing this issue required weeks of intensive work from multiple people”, the price would have to be prohibitively high.
More generally, software is really, really expensive to produce and maintain. The economics only work at scale, in particular for B2C. (Maybe AI will change that, if it becomes more reliable.)
For many large companies or even teams, there exists a class of bugs / issues / features where dropping 5-10k on a bounty is extremely cost efficient compared to working around the issue or internal development. That might not fund development outright, but at worst it would point out the features people want and serve to inform what to work on next. I think there are a couple reasons why that is not prevalent. Most important one is that highly compensated enterprise teams that would benefit the most from placing bounties tend to avoid software that is lacking features or has bugs. Secondary is not implemented here ego and general disconnect between people in the trenches that know what needs to be done and people controlling ability to place bounties.
Imagine FAANG assigning $500 per engineer per year to allocate to feature / bug bounties.
Most larger companies would probably find it way easier and more sensible to contract with some outside consultancy to work on these issues than just posting a random bounty, even if the latter might potentially be cheaper. See Google Summer of Code projects for a very practical example of how "just pay randos to work on issue X for cheap" can quite often end up in failure.
Eh, I think you're underestimating some people perseverance.
You generally only need multiple people for timely action, and it usually even slows you down (from the perspective of total hours spent)
Like 2k bug bounty? I guarantee you some people would be willing to spend a lot of time for that. But yeah, people which are gainfully employed and have a decent salary - likely not.
People will have fun spending their free time on such projects. But it’s virtually impossible to turn it into “an actual thriving business model” that people can make a living on.
lt could become some sort of leetcode final boss and/or something that you can put on your resume.
And also scarce skills.
Did you realize that you didn't include 'open source' in your statement? This is exactly what the desktop OS makers -Microsoft and Apple- do every single day. Their prices are mostly B2B and therefore hidden, but there is a steady income for each person involved in making the fix.
and yet, Microsoft Teams is a total trash fire full of bugs that users hate. So something is broken (Teams. It's Teams that is busted).
it's the management structure that's broken. Plenty of decent engineers around microsoft who could fix it, plenty of customer and enterprises willing to pay, but they are not allowed to work on it because of prioritization bullshit, allegedly they could get more money elsewhere
That's literally the issue, management by KPI frameworks
I think it has more to do with bundling reducing the need to compete to zero. Change that and the economics of competition would take over and the changes would get prioritized but nobody at Teams needs to sell a single license, so the priorities become the bs like internal status and visibility and not product success.
How many companies have Teams for basically free with their 365 license but still pay for Slack? The marginal value of Teams is nearly zero.
There is also a matter of selective effort by staff senior enough to make their own choices. Many SDE3 (or whatever MS equivalent is) wouldn’t want to be associated with a dumpster fire product like Teams.
I have used it every day for past 3-4 years. What bugs? I don’t love it but I don’t hate it either. I don’t understand the Teams hate
If you had made the same complaint about Win11 and you wouldn't be so far off. Microsoft is great at driver support which is the subject at hand.
The problem is one-offs don't make steady, predictable, recurring revenue. Owning a consulting business is hard: you have to have customers waiting.
I wish there was regulation that you have to sell and maintain a working product, so that open source devs don't have to waste their time fixing proprietary products.
It looks like these laptops are usually sold with Windows; are you saying that every manufacturer should be obligated to develop drivers for every software which is theoretically compatible with it? Or are you just saying that we need even more caveats in the interminable EULAs we all just click through?
I think that 2k is really really cheap for the expertise in kernel development
It is, but it's amazing how cheap kernel expertise is relative to comparable experience in other specialties like frontend.
there are a lot more kernel programmers than kernel work
But also lots of kernel developers work for free, so the average price of their work is very low
"Lots" is a relative term, but the overwhelming majority of kernel developers are employed and usually do kernel work as part of their job (usually at least ~80% but it could be argued as high as 97% depending on how you interpret the breakdown done by LWN of each release[1]).
And I would guess that most of the kernel devs who are "working for free" are doing the stuff they personally enjoy and find satisfaction in working on, because it's a hobby -- so many of them are probably not interested in fixing random bugs for cash either.
For small stuff, the cost is just going to be too much for people to want to pay it. This bug had a $1900 bounty attached. Let's put the cost of one software engineer (salary plus overheads) at $200,000 a year, which I think is an underestimate. That's $3850 a week, so unless your bug can definitely be fixed (including getting any necessary hardware, investigation, fixing, code review overhead, etc) in two or three days it doesn't pay. And if it could obviously be done in two days then it's likely somebody would have already done that.
The above back of envelope maths ignores the overheads of interacting with the people who posted the bounties to get them to agree to pay up, and of the cost overruns on the class of bugs that look like two day fixes but take two weeks.
$200k is one expensive software engineer. On average, you can get people to work for much less.
I assumed the commonly cited 2x markup, so that would be a $100k salary, which is less than various websites say is the average US software dev salary. You could probably find cheaper elsewhere in the world, but even if you cut the salary in half that's still "bug must be doable in a week", which isn't going to cover many of the bugs people will care about.
I believe that the $200k figure was meant to express what such a person might cost the company, not what that person would be paid as salary.
(And it's just a placeholder. $200k seems like it's at least in the direction of the right ballpark.)
Paying for software developers is really weird. State governments for example struggle to pay for a FTE that makes $140k. But they can pay me over $200/hour for consulting services for multiple years. The technical FTE employees that they have generally aren't qualified to evaluate their consulting needs so you get multi-million dollar contracts with very little actual oversight. I was really impressed with the folks I was working with at this particular state government and looked into what it would look like if I joined them full time as a FTE technology leader. I would have to take almost a 50% pay cut. The top senior IT position that oversees all of the state resources makes 70% of what I do. It's crazy. Unless you're working in medicine or sports, government pay sucks.
I've seen similar but less extreme examples play out in the private sector. 16 year senior architect making less than freshly hired software dev that was just an intern within the same company. Software developer pay is largely based on what you're demanding. In a lot of companies, there is a wide range of pay for folks doing literally the same job. They will hire a dev at $180k because that dev wouldn't go lower and turn around and push back to get another dev at $120k for the same level of unproven experience.
200k is a fairly high salaried software eng in expensive markets, a bounty program like this would be open worldwide and many people would be willing to work for a fraction of that, quality control is another concern but take a look at prices on sites like upwork and bids for this type of work and realize 200k is nowhere near the lower baseline.
$200k in cost to the company is a lot different than $200k in salary. It probably relates to someone making like $140k, depending on the various tax rates.
also, don't forget to include QA and release management overhead, as well as projectmanagement etc...
the 60k buffer probably just covers the salaries of the multiple layers of management and facilities (building, cleaning...)
You are forgetting that typically many users want a bug fixed.
$200k is on the extreme high-end of software engineers. For example in eastern europe $30k is normal. And that's not even the floor. You can go to india or africa to get even cheaper. The problem with this bug bounty though is that it requires pretty rare expertise. It's not a "throw any developer at it" type of thing.
Yeah, you'd want some sort of micro-kickstarting website where users can pool money that goes into paying for some fix or feature if the committed money crosses a threshold.
People spam the most minimal viable patch to collect the bounty and move on. And these days they are sending an AI slop solution. It doesn’t promote good code like actually hiring someone.
I'd gladly pay a couple hundred to have Swift-like optionals in Godot's GDScript, among other things that are just a pain to convince all the random idiots on their official spaces of, but GitHub doesn't have a way to offer that :(
I think the real issues are attributing work, and fear of doing a ton of work only to be pipped at the post.
The paperwork.
There is a common problem with Realtek ALC3306 on Linux (Kernel Bug 213159). This affects many Lenovo laptop models. For example, my fairly old Legion S7 15IMH5 laptop also does not work.
I'm not willing to pay $1000 for a fix (it's easier for me to buy a new laptop that will work with Linux), but $100 is probably okay. :)
And the person who did the implementation, Lyapsus, did it without access to the hardware?? https://github.com/nadimkobeissi/16iax10h-linux-sound-saga/i...
That thread is a fun (though frustrating for them!) conversation to read through.
After about a hundred back-and-forths getting the guy with the actual hardware to try different commands, I was thinking to myself man, maybe he should just give him remote access to work on the target PC, this is torture for both of them. And then I see him comment:
> Honestly I'm thinking of this and maybe something insane like organizing ssh access or something to quit torturing Nadim with building and rebooting all the time
And Nadim replies:
> Haha, sorry, but there's no way I'm giving you SSH access!
> I’m fine with continuing with tests!
Which is fair enough! But was funny to see right when I was thinking the same thing. Great perseverance from both of them.
Was slightly disappointing they they moved off GitHub to Discord eventually so after all that, we miss the moment of them actually getting it working!
I guess this here is what the title is describing: https://github.com/nadimkobeissi/16iax10h-linux-sound-saga/b...
Title should be: $2000 Bug Bounty to Fix the Lenovoe Legion Pro 7 16IAX10H's Speakers on Linux
Oh it's written by Nadim Kobeissi, such a huge fan of his work didn't expect him see him here
In the README:
> Approximately 95% of the engineering work was done by Lyapsus. Lyapsus improved an incomplete kernel driver, wrote new kernel codecs and side-codecs, and contributed much more. I want to emphasize his incredible kindness and dedication to solving this issue. He is the primary force behind this fix, and without him, it would never have been possible.
> I (Nadim Kobeissi) conducted the initial investigation that identified the missing components needed for audio to work on the 16IAX10H on Linux. Building on what I learned from Lyapsus's work, I helped debug and clean up his kernel code, tested it, and made minor improvements. I also contributed the solution to the volume control issue documented in Step 8, and wrote this guide.
For those wondering:
> Sincere thanks to everyone who pledged a reward for solving this problem. The reward goes to Lyapsus.
If only people would do these bounties for ITE sensor hubs.
Where are LLMs now?
They're not useful for fixing things like this. Only frontend React.js
> Only frontend React.js
Good suggestion, but I discovered that React was not able to fix my Linux kernel, either, for some reason.
Have you tried asking an llm to use react to fix your Linux kernel?
Could an AI agent kidnap Torvalds and force him to do it?
Vibe coder: "ChatGPT, please fix the Lenovo Legion Pro 7 16IAX10H's Speakers on Linux"
Hal3000: "Great request—here is your React version 20XX* TODO list"
*20XX is a year+ old version of React
if there were data sheets available I expect they actually could do a bunch of the work here.
Interestingly the Awinc aw88399 smart amplifier chip at the core of this issue allegedly got supporting in 6.7, in 2023. https://www.phoronix.com/news/Linux-6.7-Sound
I have a couple old-ish Samsung Galaxy Book x86 tablets that have a similar issue, that I have never quite goaded myself into trying to reverse engineer. I'd love some better material on trying to reverse engineer windows drivers: presumably maybe running windows in qemu with some kind of intercepting pass through?
Patching up the kernel to get some sound coming out of the speakers.. Very on brand for Linux.
How do you know somebody has no idea what they're talking about? They'll tell you.
deep
Show us on the doll where they’re wrong.
I'm guessing it's the fact that linux has in-tree drivers, so you necessarily need to "patch the kernel" in order to write/fix a driver for a non-standard compliant device?
I can’t see any blocker to publishing this as a prebuilt kernel module honestly.
For driver developers the above where you rebuild the kernel is a necessary step in developing the driver but now the above is done someone should make the trivial next step to make this into a prebuilt kernel module which are trivial to install for end users with no rebuild/reboot required. (I have built kernel modules before but I don’t have this laptop myself, sorry!).
How else is that supposed to work?
You either fix a driver in the kernel or a driver outside the kernel, it's not going to make that big of a difference to the person who has to fix it.
The difference is that the end user doesn't have to do it. Someone else is going to do it. Just like it is on Windows.
That's how it works in Linux land for two reasons. One, drivers live in the kernel (roughly). Two, Linux is aftermarket for many hardware, in which cases there's hardware first, then the support.
I agree. I can’t see any reason this couldn’t be packaged as a prebuilt kernel module so end users can trivially install it. The instructions and code here can be used to build the kernel module.
I don’t have this laptop but have built kernel modules in the past to give context. It’s a tiny step to publish this as a kernel module so end users can trivially install this (this reduces the instructions to downloading one file, running one command, with no reboot or rebuild needed) so it’s quite reasonable to call this out and ask someone to do it.
It’s a bit like publishing a windows driver as raw source code. Great work but there’s no reason not to ship the prebuilt driver right?
Some device classes can be supported in userspace because no matter how an adversarial driver might get the device to misbehave, it cannot possibly break the kernel's security model. This might even apply to some audio devices, depending on how exactly they're hooked up to the rest of your system. But the more typical devices, especially those in your average SoC and those connected to a PCIe bus or the like, have full privileges within the system and will need kernel-level support for the foreseeable future.
Kernel modules absolutely run in kernel space though.
I’ve literally written kernel modules for high speed networking devices that have full access to the memory bus and enumerate pci devices. There’s no userspace or kernel space question here. It’s merely a matter of someone turning this into an easily installable kernel module
Kernel modules are not going to be "easily installable" anyway because their whole purpose is to poke at kernel-internal structures that will change all the time as the kernel evolves. With source code, you'll hopefully get notified if there is breakage - the module fails to build and you need to forward-port it to the current kernel.
They have great stability between kernels by design. Better than Windows dll based drivers IMHO.
As someone who actually writes drivers I'm a little frustrated at this whole thread with people claiming Linux drivers have to be distributed this way.
Kernel modules exist for a reason, literally to allow end users as easy and as forwards compatible of a way to install drivers as windows dll based drivers. This whole thread has a lot of know nothings chiming in if I'm blunt.
Here is another[1] fine read that was post recently. Author fixed media button issue for old 2005 Fujitsu keyboard and pushed it to kernel.