Tuesday, March 24, 2015

Watson and open source software

For all the sophistication and advanced algorithms that go into Watson, the Watson system is largely based on freely available open source software. Apache UIMA, Apache Lucene, Indri, and SPARQL and the Jena Toolkit are all open source software. Specifically, Watson uses open source software as basic frameworks but builds on proprietary algorithms as components that plug in to the framework. Open source software has come a long way to be a key enabler of software innovation.


Watson's way of reasoning is to generate hypotheses (that is, candidate answers) from a large body of documents, as opposed to from per-conceived theories as humans typically do. In fact, a major trend in scientific research is to “mine” discoveries from data. While Watson is trying to emulate human intelligence, humans seem to think more like Watson too! See the Resources section for an excellent article in Wired magazine for more on this issue.The workflow Watson goes through to answer a question