Ruby’s Vulnerability Handling Debacle

Max Caceres | July 1st, 2008 | Filed Under: Development, Disclosure

A bit over a week has now gone by after critical vulnerabilities in ruby’s runtime were announced and a couple of interesting developments occurred:

  • The new official ruby releases which include patches for the vulnerabilities actually break critical functionality. In fact, some of the most popular ruby frameworks segfault with any of these releases.

  • It turns out that the breakage brought by the new releases is not due to the security fixes for Drew Yao’s vulnerabilities, but to other changes that were merged into each release’s branch.

  • No response or timeline from ruby’s maintainers to produce a good release has been made public.

  • In an attempt to provide a workable middle ground, third-parties create their own patches, and so do distributors. Some start from a patched release and then work back through the changes until tests pass, others take the opposite approach and start with a last “known-good” release against which they apply selected patches to plug the security holes. Needles to say, the results of each approach are not equivalent.

Developers relying on any of these frameworks are then faced with a difficult choice: wait an indeterminate amount of time for a good release from the official maintainers (and remain vulnerable in the meantime), or apply a patch from a third-party which may not plug all the security holes, or may include unforeseen bugs. This is clearly a problem. In the meantime maintainers have to scramble to get ruby back to a good state that actually plugs the holes while people complain. Everyone loses.

To some extent this is the other side of security response we sometimes advocate for and now we got a taste of: we want our patches and we want them now! And we got them, but our apps don’t work.

The beauty of open-source is not just that we get stuff for free but that we also get a privileged view into the processes around building and maintaining wildly popular software. This is a great opportunity for us as both security and software people to look into why these processes failed and hopefully learn something. Here are my top 3 takeaways:

Keep a “last-known-good” branch to apply critical patches to. The fact that patches had to be applied to unreleased versions of the code means that it is impossible to measure the impact of the change. IMHO this is the single most important mistake behind this state of affairs. This is almost akin to not using version control. Security problems will pop up unpredictably outside of any planned release schedule, and often require a rapid response. End-users may not be able to afford upgrading to a release with new features and even bug fixes every time this happens. It is then highly desirable that a new release which only includes the necessary security patches is made available, to minimize the risk of getting pwned by a new bug or some defiant kid. Are you ready to release critical patches to all the deployed versions of your software even as you are working on its next version?

Open line of communication with your community. It is hard to believe that passing Rails’ tests isn’t an acceptance requirement for every Ruby release, since Rails is the driving force behind Ruby’s adoption. The success of any platform depends on the applications it supports, without the apps the platform doesn’t matter. Microsoft learnt this a long time ago, and that’s why you’ll see them test third-party apps with new releases and security patches, even though technically is not their responsibility. Who are the relevant members of your community? Are you involving them in your release process?

Be more open about vulnerability details. When the vulnerabilities were originally announced very few details were made available (CVE entries for the vulns have been updated with more details since then, though there’s still no entry for the Bignum bug). Because ruby is an open-source project anybody can go and access their source control system to figure out what was actually patched. The fact that you can should be no news to readers of this blog, as people have been doing the same with binary patches for quite some time. Having direct access to not only source but the full source control system is a luxury! Given that the official release didn’t work it was important to isolate the security fixes, and to easily test if 3rd party patches were effective against the flaws. In fact, one of the test cases showed that the official release missed one of Drew’s patches. Lack of details just made it harder for the people trying to help. Do you have a policy about disclosing vulnerability details in your software? Would partners and customers have enough information to evaluate the impact of your patch to their infrastructure?

Just yesterday Apple released a security update which includes fixes for the ruby vulnerabilities. The patched build appears to deal correctly with all test cases, including the string concatenation case, and with Rails’ tests. The official ruby downloads still have issues with both.

Viewing 12 Comments

Trackbacks

close Reblog this comment
Powered by Disqus · Learn more
blog comments powered by Disqus