Measuring the Value of Corporate Open Source Contributions
This is part 1 of a series on the value of an open source software development group for companies that rely on open source technology.
If you’ve worked in a corporate development environment, you certainly understand that metrics are everything. If you’re doing development, you are probably familiar with the feeling that metrics aren’t perfect. I can’t count how many times I’ve heard, “Well, I’m measured on X because it generates a number, but let me tell you the real story…”
Certain things are both meaningful and easy to measure such as the number of conference talks accepted and presented, internal training sessions delivered, or other employees that are mentored. But what do you do about code?
What Does it Mean to Measure the Value of Your Open Source Contributors?
As hard as it is to measure an individual developer’s code contributions using a standardized set of statistics, it can be even harder to measure the value of a company’s open source contribution strategy. This is one of those things that everybody knows has value (we all know it’s a lot), but how do you quantify it?
One of the first things we have to get comfortable with is the arbitrary nature of valuing contributions. A single line that fixes a buffer overflow is arguably more valuable than a single line of documentation, but by how much? We can argue this endlessly, but it very quickly becomes a problem of bikeshedding, where arguments about measurements become a distraction from development itself. Realistically, aside from giving people like me something to think about, there’s not a lot of value in arguing the semantics…
However, one valuable measurement is how much of your code has landed in an upstream project. In our case, this is the single most important metric for the members of the Samsung Open Source Group. As a result, we specifically hire high-performance maintainers and committers.
How we do it
If you want to learn about the specific methodology we use to get real and meaningful statistics on open source contributions, continue reading this post on the Samsung Open Source Group blog »
"A single line that fixes a buffer overflow is arguably more valuable than a single line of documentation, but by how much?" - I am not sure I agree with this statement.Not only if it is correct, but even if the comparison is useful. A single line of documentation might be the starting point of a new member of the community that would ultimately become a core part of the project over years, while a single code fix could be the only fix that person ever contributes. How do you measure that? Also, in some cases, people would just contribute code or just contribute documentation - therefore there is little value on trying to decide what is most valuable. All projects need both... I fear this is a slippery slope
Thank you, Brian, for posting this. The value of open source code is incredibly underestimated and underrated. Each time you use a Linux kernel, browse with Firefox, launch a ssh session with PuTTY, or create SSL certificates or connections using openSSL, you tap into just a fraction of what's been made available. As a university, we both value and strive to also make some contributions for the benefit of the open-source community as a whole. Bravo for raising the awareness!
Glad you liked it, Ravi D.. As with any metrics, there is a broader personal story to be told around the actual numbers that will vary from project to project, which helps complete the picture for a given individual. So, while they may not tell the whole story on their own, we think they're a reasonably accurate measure of what Samsung's Open Source Group is working to achieve in the open source community.
Great Articles Brian Warner ! There are always challenges on measuring the value of codes - both in open source paradigm and in proprietary settings. Patch count and line committed are definitely a good metrics. Because each metrics are unique and contribute differently, its hard to make good judgement on one over another. Definitely, if you write well structured and standard codes, you might not need a detail documentation; Conversely if you have a good documentation - people will embrace your codes no matter how complex it is. Great piece of article, thanks for sharing!